Depth extraction of three-dimensional objects in space by the computational integral imaging reconstruction technique

A novel approach to extract the depth data of 3D objects in space by using the computational integral imaging reconstruction (CIIR) technique is proposed. With elemental images of 3D objects captured by the CCD camera through a pinhole array, depth-dependent object images can be reconstructed on the output plane by the CIIR technique. Only the images reconstructed on the output planes where 3D objects were located are clearly focused; so the depth data of 3D objects in space can be extracted by discriminating these focused output images from the others by using an image separation technique. A feasibility test of the proposed CIIR-based depth extraction method is carried out, and its results are discussed as well.     

Depth extraction of three-dimensional objects using block matching for slice images in synthetic aperture integral imaging

We describe a computational method for depth extraction of three-dimensional (3D) objects using block matching for slice images in synthetic aperture integral imaging (SAII). SAII is capable of providing high-resolution 3D slice images for 3D objects because the picked-up elemental images are high-resolution ones. In the proposed method, the high-resolution elemental images are recorded by moving a camera; a computational reconstruction algorithm based on ray backprojection generates a set of 3D slice images from the recorded elemental images. To extract depth information of the 3D objects, we propose a new block-matching algorithm between a reference elemental image and a set of 3D slice images. The property of the slices images is that the focused areas are the right location for an object, whereas the blurred areas are considered to be empty space; thus, this can extract robust and accurate depth information of the 3D objects. To demonstrate our method, we carry out the preliminary experiments of 3D objects; the results indicate that our method is superior to a conventional method in terms of depth-map quality.     

Integral imaging using phase-only LCoS spatial light modulators as Fresnel lenslet arrays

We present a digital integral imaging system. A Fresnel lenslet array pattern is written on a phase-only LCoS spatial light modulator device (SLM) to replace the regular analog lenslet array in a conventional integral imaging system. We theoretically analyze the capture part of the proposed system based on Fresnel wave propagation formulation. Because of pixelation and quantization of the lenslet array pattern, higher diffraction orders and multiple focal points emerge. Because of the multiple focal planes introduced by the discrete lenslets, multiple image planes are observed. The use of discrete lenslet arrays also causes some other artifacts on the recorded elemental images. The results reduce to those available in the literature when the effects introduced by the discrete nature of the lenslets are omitted. We performed simulations of the capture part. It is possible to obtain the elemental images with an acceptable visual quality. We also constructed an optical integral imaging system with both capture and display parts using the proposed discrete Fresnel lenslet array written on a SLM. Optical results when self-luminous objects, such as an LED array, are used indicate that the proposed system yields satisfactory results.     

Microlens arrays for integral imaging system

When designing a system capable of capturing and displaying 3D moving images in real time by the integral imaging (II) method, one challenge is to eliminate pseudoscopic images. To overcome this problem, we propose a simple system with an array of three convex lenses. First, the lateral magnification of the elemental optics and the expansion of an elemental image is described by geometrical optics, confirming that the elemental optics satisfies the conditions under which pseudoscopic images can be avoided. In using the II method, adjacent elemental images must not overlap, a condition also satisfied by the proposed optical system. Next, an experiment carried out to acquire and display 3D images is described. The real-time system we have constructed comprises an elemental optics array with 54 H x 59 V elements, a CCD camera to capture a group of elemental images created by the lens array, and a liquid crystal panel to display these images. The results of the experiment confirm that the system produces orthoscopic images in real time, and thus is effective for real-time application of the II method.     

Generation of three-dimensional integral images from a holographic pattern of 3-D objects

We present a novel approach for generating three-dimensional (3-D) integral images from a fringe pattern of 3-D objects. A recorded hologram of 3-D objects is segmented into a number of subholograms. Then, different views of 3-D objects are reconstructed from them because each subhologram has its own perspective of 3-D objects in the recording process. These locally reconstructed images can be rearranged as the same subimage array of the conventional integral-imaging system and transformed into virtually picked-up elemental images of 3-D objects. From this newly generated elemental image array, 3-D images could easily be reconstructed by using a white light. Experiments with a 3-D test object have been performed and the results have been presented.     

Magnification of 3D reconstructed images in integral imaging using an intermediate-view reconstruction technique

A new integral imaging (II) system that can magnify 3D reconstructed images by employing an intermediate-view reconstruction technique (IVRT) is proposed in which the number of the elemental images obtained from a one-step pickup process can be computationally increased by use of an IVRT without mechanical movement and a long multistep pickup process. To show the feasibility of the proposed II system, some optical experiments on the magnification of 3D reconstructed images with a real 3D object have been carried out and results are presented.     

Effect of illumination in an integral imaging system with large depth of focus

We present the characteristics of integral imaging systems with large depth of focus (DOF) by use of two kinds of illumination: plane illumination and diffusing illumination. For each system, we perform ray analysis based on ray optics. To check the visual quality through optical experiments, we use an average image of observed images picked up at various positions within a large DOF. The synthesized elemental images for a three-dimensional (3-D) object with two character patterns were displayed in an optical system and its reconstruction experiments are performed. Experimental results show that use of diffusing illumination can improve visual quality of reconstruction 3-D images in depth-priority integral imaging.     

Visibility-enhanced reconstruction of three-dimensional objects under a heavily scattering medium through combined use of intermediate view reconstruction, multipixel extraction, and histogram equalization methods in the conventional integral imaging system

In this paper, we propose an effective approach for reconstructing visibility-enhanced three-dimensional (3D) objects under the heavily scattering medium of dense fog in the conventional integral imaging system through the combined use of the intermediate view reconstruction (IVR), multipixel extraction (MPE), and histogram equalization (HE) methods. In the proposed system, the limited number of elemental images (EIs) picked up from the 3D objects under the dense fog is increased by as many as required by using the IVR technique. The increased number of EIs is transformed into the subimages (SIs) in which the resolution of the transformed SIs has been also improved as much as possible with the MPE method. Subsequently, by using the HE algorithm, the histogram of the resolution-enhanced SIs is uniformly redistributed over the entire range of discrete pixel levels of the image in a way that the subimage contrast can be much enhanced. Then, these equalized SIs are converted back into the newly modified EIs, and consequently a visibility-enhanced 3D object image can be reconstructed. Successful experimental results with the test object confirmed the feasibility of the proposed method.     

Speckle-reduced three-dimensional volume holographic display by use of integral imaging

We propose a method to implement a speckle-reduced coherent three-dimensional (3D) display system by a combination of integral imaging and photorefractive volume holographic storage. The 3D real object is imaged through the microlens array and stored in the photorefractive crystal. During the reconstruction process a phase conjugate reading beam is used to minimize aberration, and a rotating diffuser located on the imaging plane of the lens array is employed to reduce the speckle noise. The speckle-reduced 3D image with a wide viewing angle can be reconstructed by use of the proposed system. Experimental results are presented and optical parameters of the proposed system are discussed in detail.     

High-Resolution Digital Integral Photography by use of a Scanning Microlens Array

We suggest what we believe is a new three-dimensional (3-D) camera system for integral photography. Our method enables high-resolution 3-D imaging. In contrast to conventional integral photography, a moving microlens array (MLA) and a low-resolution camera are used. The intensity distribution in the MLA image plane is sampled sequentially by use of a pinhole array. The inversion problem from pseudoscopic to orthoscopic images is dealt with by electronic means. The new method is suitable for real-time 3-D imaging. We verified the new method experimentally. Integral photographs with a resolution of 3760 pixels x 2560 pixels (188 x 128 element images) are presented.     

Geometrical effects of positional errors in integral photography

The effects of misarrangement of elements (elemental lenses and elemental images) that construct three-dimensional (3-D) images in integral photography are presented. If the lens arrays of the capturing system and the display system are not aligned accurately, positional errors of elements may occur, causing the 3-D image to be reconstructed in an incorrect position. The relation between positional errors of elements and the reconstructed image is derived. As a result, it is shown that a 3-D image is separated by local positional errors and blurred by global positional errors. In both local and global positional errors, 3-D images reconstructed far from the lens array are greatly affected.     

Wide-viewing integral three-dimensional imaging by use of orthogonal polarization switching

A wide-viewing integral three-dimesional (3D) imaging system that adopts orthogonal polarization switching is proposed and demonstrated. In our scheme,the polarizing sheet attached to the lens array and the orthogonal polarization switching of the elemental image array perform elemental lens switching. The experimental results document that the viewing angle becomes remarkably wider than that of the conventional method. The distinguishing feature of our system is that it requires no mechanical moving part. In addition, because a commercially available polarization shutter screen is used for electrical switching, it is easy to implement this as a practical system. We believe that the proposed method facilitates the practical use of this wide-viewing integral 3D imaging system.     

Three-dimensional integral display using plastic optical fibers

A novel approach to an integral imaging system using a pliable plastic optical fiber array is proposed. The proposed system has the advantage that it can utilize a light source for three-dimensional (3D) images at an arbitrary location because the point light sources are formed by the plastic fiber array with flexible optical paths. Two-dimensional images can also be expressed in the proposed system. The light efficiency of this system is high compared with previous point light source array integral imaging systems. The feasibility of the proposed method is explained and demonstrated with experiments.     

Effect of different traversal schemes in integral image coding

Integral imaging (InIm) is a highly promising technique for the delivery of three-dimensional (3D) image content. During capturing, different views of an object are recorded as an array of elemental images (EIs), which form the integral image. High-resolution InIm requires sensors with increased resolution and produces huge amounts of highly correlated data. In an efficient encoding scheme for InIm compression both inter-EI and intra-EI correlations have to be properly exploited. We present an EI traversal scheme that maximizes the performance of InIm encoders by properly rearranging EIs to increase the intra-EI correlation of jointly coded EIs. This technique can be used to augment performance of both InIm specific and properly adapted general use encoder setups, used in InIm compression. An objective quality metric is also introduced for evaluating the effects of different traversal schemes on the encoder performance.     

Generic camera model and its calibration for computational integral imaging and 3D reconstruction

Integral imaging (II) is an important 3D imaging technology. To reconstruct 3D information of the viewed objects, modeling and calibrating the optical pickup process of II are necessary. This work focuses on the modeling and calibration of an II system consisting of a lenslet array, an imaging lens, and a charge-coupled device camera. Most existing work on such systems assumes a pinhole array model (PAM). In this work, we explore a generic camera model that accommodates more generality. This model is an empirical model based on measurements, and we constructed a setup for its calibration. Experimental results show a significant difference between the generic camera model and the PAM. Images of planar patterns and 3D objects were computationally reconstructed with the generic camera model. Compared with the images reconstructed using the PAM, the images present higher fidelity and preserve more high spatial frequency components. To the best of our knowledge, this is the first attempt in applying a generic camera model to an II system.     

Active contour segmentation by use of a multichannel incoherent optical correlator

We describe an optoelectronic incoherent multichannel processor that is able to segment an object in a real image. The process is based on an active contour algorithm that has been transposed to optical signal processing to accelerate image processing. This implementation requires exact-valued correlations and thus opens attractive perspectives in terms of optical analog computation. Furthermore, this optical multichannel processor setup encourages incoherent processing with high-resolution images.     

Gradient-index lens-array method based on real-time integral photography for three-dimensional images

Because a three-dimensional (3-D) autostereoscopic image can be seen from a desired viewpoint without the aid of special viewing glasses, integral photography (IP) is an ideal way to create 3-D autostereoscopic images. We have already proposed a real-time IP method that offers 3-D autostereoscopic images of moving objects in real time by use of a microlens array and a high-definition television camera. But there are two problems yet to be resolved: One is pseudoscopic images that show a reversed depth representation. The other is interference between the element images that constitute a 3-D autostereoscopic image. We describe a new gradient-index lense-array method based on real-time IP to overcome these two problems. Experimental results indicating the advantages of this method are shown. These results suggest the possibility of using a gradient-index lens array for real-time IP.     

Single-lens single-image incoherent passive-ranging systems

We introduce a new system for single-lens single-image incoherent passive ranging. The only a priori object information this system requires is that the objects to be ranged must possess a low-pass spatial frequency spectrum. Physically, this system for passive ranging is a standard optical imaging system that is customized with a special-purpose optical mask or filter. Analytically, this optical mask customizes the transfer function of the optical system in such a way that objects form images that contain range-dependent information. This range-dependent information lies in the spatial spectrum nulls or zeros of the image.     

Multidimensional optical sensor and imaging system

We describe a multidimensional optical sensor and imaging system (MOSIS). Using a time-multiplexing, polarimetric, and multispectral imaging system, we are able to reconstruct a fully integrated multidimensional scene. Image fusion is used to integrate the multidimensional images. The fused image contains more information than the single two-dimensional and three-dimensional (3D) images. The multidimensional imaging system utilizes polarimetric imaging, multispectral imaging, 3D integral imaging with time and space multiplexing, and 3D image-fusion techniques to reconstruct the multidimensionally integrated scene. Optical experiments and computer simulations are presented.     

Nanophotonic three-dimensional microscope

Three-dimensional (3D) optical microscopy based on integral imaging techniques is limited mainly by diffraction effects and the pitch of the microlens array used to sample the specimen. We integrate nanotechnology to the integral imaging technique and demonstrate a nanophotonic 3D microscope, where a nanophotonic lens array is used to finely sample the specimen. The resolution limitation due to diffraction is reduced by capturing images before the diffraction effects predominate and hence overcomes the bottleneck of achieving high resolution in an integral imaging 3D microscope.