A high‐resolution autostereoscopic display system with a wide viewing angle using an LCOS projector array

Abstract— This study develops an autostereoscopic display based on a multiple miniature projector array to provide a scalable solution for a high‐resolution 3‐D display with large viewing freedom. To minimize distortion and dispersion, and to maximize the modulation transfer function (MTF) of the projection image to optimize 3‐D image quality, a dedicated projection lens and an accurate six‐axis adjusting platform for the miniature projector were designed and fabricated. Image‐blending technology based on a lookup table was adopted to combine images from 30 miniature projectors into a seamless single image. The result was a 35‐in. autostereoscopic display with 60 views ata 30° viewing angle, 90° FOV, and large range of viewing distance. The proposed system exhibits very smooth motion parallax when viewers move around in front of it.     

Viewing angle enhanced integral imaging display based on double‐micro‐lens array

A viewing angle enhanced integral imaging display, which consists of a double microlens array, and a display panel is proposed. The double microlens array includes a convex microlens array and a concave microlens array. The display panel is used to display original elemental image array. The convex microlens array, located near the display panel, is used to provide a virtual elemental image array for the concave microlens array. The concave microlens array, located far away from the display panel, is used to display integral images with the virtual elemental image array. Compared with the original elemental image, the pitch for each virtual elemental image is magnified by the corresponding convex microlens. As a result, the viewing angle is expanded. Simulations based on ray‐tracing are performed and the results agree well with the theory.     

Seamless ultrathin rear projection display

A new architecture for a thin (2‐cm depth) rear projection display is described. In order to achieve this small depth, a very high density of rear projectors is used. Three prototype displays using rear projectors on both 5‐ and 2‐cm pitch arrays are described. The displays can achieve an effective screen pixel pitch of as small as 0.5 mm, which makes this technology competitive in terms of resolution with fine pitch LED displays; however, orders of magnitude fewer LEDs are required: Each rear projector requires only one white LED and a color liquid crystal light modulator. In the three prototypes, the projector light modulators utilize 101‐cm (40 in.), 80‐cm (31.5 in.), and 60‐cm (24 in.) diagonal liquid crystal display glass. To minimize cost, no lenses are utilized for the rear projectors. An RGB LED array may augment the projector array, which provides a low resolution component of the image onto which the high resolution component is superimposed by the projector array. Edge gaps between active areas on adjacent LCD glass units are completely eliminated by the rear projection approach enabling low profile wall‐size seamless displays. Display contrast depends on rear projection screen design.     

Implementation of polarization‐multiplexed tiled projection integral imaging system

Abstract— A 40‐in. tiled projection integral imaging system has been implemented, adopting a polarization‐multiplexing technique. The system is composed of two full‐high‐definition (HD) projectors, a time‐varying polarizer, a polarization preserving screen, polarization films, a lens array, and a control unit. An elemental image set is projected using two full‐HD projectors to enhance the resolution of the system. The viewing region of the system is increased by using a polarization switching method. The polarization state of the elemental image set is changed by the time‐varying polarizer, and the elemental image set is diffused by the polarization preserving screen. The elemental image set with a preserved polarization state forms a three‐dimensional image with increased viewing angle by the integration of a lens array with polarization films. A 60‐in. tiled projection integral imaging system was also demonstrated using four full‐HD projectors.     

Real‐Time Human Shadow Removal in a Front Projection System

When a person is located between a display and an operating projector, a shadow is cast on the display. The shadow on the display may eliminate important visual information and therefore adversely affect the viewing experiences. There have been various attempts to remove the human shadow cast on a projection display by using multiple projectors. While previous approaches successfully removed the shadow region when a person moderately moves around or stands stationary in front of the display, there is still an afterimage effect due to the lack of consideration of the limb motion of the person. We propose a new real‐time approach to removing the shadow cast by a person who dynamically interacts with the display, making limb motions in a front projection system. The proposed method utilizes a human skeleton obtained from a depth camera to track the posture of the person which changes over time. A model that consists of spheres and conical frustums is constructed based on the skeleton information in order to represent volumetric information of the person being tracked. Our method precisely estimates the shadow region by projecting the volumetric model onto the display. In addition, employment of intensity masks that are built based on a distance field helps suppress the afterimage of the shadow that appears when the person moves abruptly. It also helps blend the projected overlapping images from different projectors and show one smoothly combined display. The experiment results verify that our approach removes the shadow of a person effectively in a front projection environment and is fast enough to achieve real‐time performance.     

Toward the light field display: autostereoscopic rendering via a cluster of projectors

Ultimately, a display device should be capable of reproducing the visual effects observed in reality. In this paper we introduce an autostereoscopic display that uses a scalable array of digital light projectors and a projection screen augmented with microlenses to simulate a light field for a given three-dimensional scene. Physical objects emit or reflect light in all directions to create a light field that can be approximated by the light field display. The display can simultaneously provide many viewers from different viewpoints a stereoscopic effect without head tracking or special viewing glasses. This work focuses on two important technical problems related to the light field display; calibration and rendering. We present a solution to automatically calibrate the light field display using a camera and introduce two efficient algorithms to render the special multi-view images by exploiting their spatial coherence. The effectiveness of our approach is demonstrated with a four-projector prototype that can display dynamic imagery with full parallax.     

Double‐viewing‐zone integral imaging 3D display without crosstalk based on a tilted barrier array

We propose an integral imaging (II) three‐dimensional (3D) display using a tilted barrier array and a stagger microlens array. The tilted barrier array consists of two orthogonally polarized sheets. In the stagger microlens array, the center of the microlens has p/2 shift with the elemental image along the horizontal direction, where p is the pitch of the microlens. The proposed II 3D display produces two different viewing zones and each of them is almost equal to that of the conventional II 3D display, and it has no crosstalk. We verify the feasibility of the proposed II 3D display in the simulation results.     

Automatic Registration of Multi‐Projector Domes Using a Single Uncalibrated Camera

In this paper we present a novel technique for easily calibrating multiple casually aligned projectors on spherical domes using a single uncalibrated camera. Using the prior knowledge of the display surface being a dome, we can estimate the camera intrinsic and extrinsic parameters and the projector to display surface correspondences automatically using a set of images. These images include the image of the dome itself and a projected pattern from each projector. Using these correspondences we can register images from the multiple projectors on the dome. Further, we can register displays which are not entirely visible in a single camera view using multiple pan and tilted views of an uncalibrated camera making our method suitable for displays of different size and resolution. We can register images from any arbitrary viewpoint making it appropriate for a single head‐tracked user in a 3D visualization system. Also, we can use several cartographic mapping techniques to register images in a manner that is appropriate for multi‐user visualization. Domes are known to produce a tremendous sense of immersion and presence in visualization systems. Yet, till date, there exists no easy way to register multiple projectors on a dome to create a high‐resolution realistic visualizations. To the best of our knowledge, this is the first method that can achieve accurate geometric registration of multiple projectors on a dome simply and automatically using a single uncalibrated camera.     

Image correction based on homography in stereoscopic projection display

Abstract— Stereoscopic and autostereoscopic projection‐display systems use projector arrays to present stereoscopic images, and each projector casts one parallax image of a stereoscopic scene. Because of the position shift of the projectors, the parallax images have geometric deformation, which influences the quality of the displayed stereoscopic images. In order to solve this problem, a method based on homography is proposed. The parallax images are pre‐transformed before they are projected, and then the stereoscopic images without geometric distortion can be obtained. An autostereoscopic projection‐display system is developed to present the images with and without calibration. Experimental results show that this method works effectively.     

Crosstalk‐free dual‐view integral imaging display

We propose a crosstalk‐free dual‐view integral imaging display. It is composed of a display panel, a barrier array, and a micro‐lens array. The central barrier is located at the vertical central axes of the display panel and the micro‐lens array to split the element image array and the viewing zone. Moreover, other barriers are located at the margins of the elemental images and corresponding micro‐lenses to eliminate the crosstalk. The lights emitting from the left and right half of the element image array are modulated by the left and right half of the micro‐lens array to reconstruct the right and left viewing zones, respectively. A prototype of the proposed dual‐view integral imaging display is developed, and good experimental results agree well with the theory.     

The auto‐geometric correction of multi‐projector for cylindrical surface using Bézier patches

Geometric calibration to projection images is an indispensable operation for projection‐based spatial display. In this paper, we propose a new method for correcting images generated in a computer onto a cylindrical surface accurately, which can project a high‐resolution projection image with pixels matching avoiding too much manual operation. Images waiting to be projected are pre‐warped according to the rough correspondence between projectors and physical surface. To solve the errors resulting from unexpected pixel shifts in overlap projection area, we fit the Bézier interpolation to the images and apply the optimization theory with added constraints to correct the projection image accurately. This optimization process, by taking the pixels with specific significance on the images as the basis of calculation, avoids the traditional ways of translating the control points of the Bézier surface directly. The final results achieve a completely accurate projection picture even if the projection surface shape is inaccurate and irregular. We present the details of the proposed accurate calibration algorithm and illustrate our method, which, with its scalability, can achieve perfect projection efficiently and accurately with experiments.     

Dual layered display that presents auto‐stereoscopic 3D images to multiple viewers in arbitrary positions

Dual layered display or also called tensor display that consists of two panels in a stack can present full‐parallax 3D images with high resolution and continuous motion parallax by reconstructing corresponding light ray field within a viewing angle. The depth range where the 3D images can be displayed with reasonable resolution, however, is limited around the panel stack. In this paper, we propose a dual layered display that can present stereoscopic images to multiple viewers located at arbitrary positions in observer space with high resolution and large depth range. Combined with the viewer tracking system, the proposed method provides a practical way to realize high‐resolution large‐depth auto‐stereoscopic 3D display for multiple observers without restriction on the observer position and the head orientation.     

基于多投影的多视点自动立体三维显示系统

多视点自动立体显示有望成为今后主流的三维显示技术,它是一种无需借助任何辅助观察设备的多视点、多观察区、高分辨率、显示效果逼真的三维显示方式。阐述了基于多投影的多视点自动立体显示系统的设计原理,详细地描述了系统的软硬件构架,建立了基于多投影仪和水平光学各向异性显示结构的自动立体显示样机,开发了投影仪阵列自动校准系统,提高了投影仪的校准精度,避免了因投影仪数目多而导致的繁琐的校准过程。实验结果能够给观众带来逼真的三维视觉体验。     

High‐resolution autostereoscopic 3‐D projection display with a space‐dividing iris‐plane shutter

Abstract— A high‐resolution autostereoscopic 3‐D projection display with a polarization‐control space dividing the iris‐plane liquid‐crystal shutter is proposed. The polarization‐control iris‐plane shutter can control the direction of stereo images without reducing the image quality of the microdis‐play. This autostereoscopic 3‐D projection display is 2‐D/3‐D switchable and has a high resolution and high luminance. In addition, it has no cross‐talk between the left and right viewing zones, a simple structure, and the capability to show multi‐view images.     

High‐quality direct‐view display combining multiple integral 3D images

This paper proposes a method for combining multiple integral three‐dimensional (3D) images using direct‐view displays to obtain high‐quality results. A multi‐image combining optical system (MICOS) is used to enlarge and combine multiple integral 3D images without gaps. An optical design with a simple lens configuration that does not require a diffuser plate prevents the deterioration in resolution resulting from lens arrangement errors and the diffuser plate. An experiment was performed to compare a previously developed method with the proposed method, and the latter showed a significant improvement in image quality. A method for expanding the effective viewing angle of the proposed optical design was also developed, and its effectiveness was confirmed experimentally. A prototype device of the proposed optical design was constructed using a high‐density organic light‐emitting diode (OLED) panel with 8K resolution and 1058 ppi pixel density to achieve 311 (H) × 175 (V) elemental images, a viewing angle of 20.6° in both the horizontal and vertical directions, and a display size of 9.1 in. In addition, the proposed optical design enabled making device considerably thinner, ie, with a thickness of only 47 mm.     

Advances towards high‐resolution pack‐and‐go displays: A survey

Abstract— Tiled displays provide high resolution and large scale simultaneously. Projectors can project on any available surface. Thus, it is possible to create a large high‐resolution display by simply tiling multiple projectors on any available regular surface. The tremendous advancement in projection technology has made projectors portable and affordable. One can envision displays made of multiple such projectors that can be packed in one's car trunk, carried from one location to another, deployed at each location easily to create a seamless high‐resolution display, and, finally, dismantled in minutes to be taken to the next location — essentially a pack‐and‐go display. Several challenges must be overcome in order to realize such pack‐and‐go displays. These include allowing for imperfect uncalibrated devices, uneven non‐diffused display surfaces, and a layman user via complete automation in deployment that requires no user invention. We described the advances we have made in addressing these challenges for the most common case of planar display surfaces. First, we present a technique to allow imperfect projectors. Next, we present a technique to allow a photometrically uncalibrated camera. Finally, we present a novel distributed architecture that renders critical display capabilities such as self‐calibration, scalability, and reconfigurability without any user intervention. These advances are important milestones towards the development of easy‐to‐use multi‐projector displays that can be deployed anywhere and by anyone.     

Optical simulation for cross‐talk evaluation and improvement of autostereoscopic 3‐D displays with a projector array

Abstract— Multi‐view spatial‐multiplexed autostereoscopic 3‐D displays normally use a 2‐D image source and divide the pixels to generate perspective images. Due to the reduction in the resolution of each perspective image for a large view number, a super‐high‐resolution 2‐D image source is required to achieve 3‐D image quality close to the standard of natural vision. This paper proposes an approach by tiling multiple projection images with a low magnification ratio from a microdisplay to resolve the resolution issue. Placing a lenticular array in front of the tiled projection image can lead to an autostereoscopic display. Image distortion and cross‐talk issues resulting from the projection lens and pixel structure of the microdisplay have been addressed with proper selection of the active pixel and adequate pixel grouping and masking. Optical simulation has shown that a 37‐in. 12‐view autostereoscopic display with a full‐HD (1920 × 1080) resolution can be achieved with the proposed 3‐D architecture.     

A toroidal‐lens designed structure for static type table‐top floating image system with horizontal parallax function

A new structure of horizontal parallax table‐top floating image system with toroidal‐lens optical film was developed. In this design, the circular arranged pico‐projectors limit the angular resolution of this system and display the floating image for surrounding viewing zones. In addition, the pinhole array and toroidal‐lens layer compose the optical film in the system and correspond with each other; both of them could be considered as a repeatable unit to control the spatial resolution of image. After passing through the optical film, the direction, position, shape, and divergence angle of light field could be controlled as fan ray, which has a widely scattered angle in latitude and high directivity in longitude direction. Moreover, to confirm the optical properties, the proposed structure was built in the commercially optical software, LightTools v8.3, which is widely used in the simulation of light distribution. Based on the imaging principle and the inverse light tracking method, displaying floating image with circular viewing zones would be achieved.     

Development of a parameterization image stitching algorithm for ultrashort throw laser MEMS projectors

We present a novel design of a parameterization image stitching algorithm for ultrashort throw laser MEMS projectors. The resultant method allows the use of projectors with short or even long throw ratios to achieve ultrashort throw projection through a parameterized algorithm to stitch multiple images into one single frame, alleviating the trade‐off between the resolution and frame rate in conventional laser MEMS displays. To evaluate the effectiveness, we construct a test system that consists of three off‐the‐shelf projectors with a throw ratio of 1.3 and three planar reflective mirrors. Using our method, the integrated system yields an ultrashort throw ratio of 0.26, greatly reducing the projection distance from 12 to 2.6 ft for the 120‐in. screen in diagonal. Moreover, our algorithm can be fine‐tuned to correct for image distortion, intensity variation, and edge mismatch induced by two‐axis light beam steering, thereby further improving the image quality.     

Circular camera array system for 3‐D displays based on the reconstruction of parallax rays

Abstract— A circular camera system employing an image‐based rendering technique that captures light‐ray data needed for reconstructing three‐dimensional (3‐D) images by using reconstruction of parallax rays from multiple images captured from multiple viewpoints around a real object in order to display a 3‐D image of a real object that can be observed from multiple surrounding viewing points on a 3‐D display is proposed. An interpolation algorithm that is effective in reducing the number of component cameras in the system is also proposed. The interpolation and experimental results which were performed on our previously proposed 3‐D display system based on the reconstruction of parallax rays will be described. When the radius of the proposed circular camera array was 1100 mm, the central angle of the camera array was 40°, and the radius of a real 3‐D object was between 60 and 100 mm, the proposed camera system, consisting of 14 cameras, could obtain sufficient 3‐D light‐ray data to reconstruct 3‐D images on the 3‐D display.