The Visual Computing of Projector‐Camera Systems

This article focuses on real‐time image correction techniques that enable projector‐camera systems to display images onto screens that are not optimized for projections, such as geometrically complex, coloured and textured surfaces. It reviews hardware‐accelerated methods like pixel‐precise geometric warping, radiometric compensation, multi‐focal projection and the correction of general light modulation effects. Online and offline calibration as well as invisible coding methods are explained. Novel attempts in super‐resolution, high‐dynamic range and high‐speed projection are discussed. These techniques open a variety of new applications for projection displays. Some of them will also be presented in this report.     

High‐efficiency optical system for ultra‐short‐throw‐distance projector based on multi‐laser light source

Abstract— A novel laser‐light‐source projector having the three outstanding features of high brightness, ultra‐short throw distance, and high color reproduction has been developed.These features have recently come to be required in the high‐end projector market. The technologies for the laser‐light‐source projectors fully utilize the advantages of lasers, such as high luminance, small étendue, and high color purity. By integrating a triple‐rod illumination system with a multi‐laser light source and an ultra‐wide‐angle projection system, the developed high‐efficiency optical system has achieved a brightness of 7000 lm and a throw ratio of 0.28 with an image size of 100–150 in. Another new technology, laser color processing (LCP), has offered vivid color reproduction which has a color gamut that is up to 180% wider than the BT.709 standard without appearing unnaturally colored. Furthermore, a speckle suppression effect produced by the multi‐laser light source has been demonstrated. In this paper, an overview of these newly developed technologies that are used in the novel laser‐light‐source projector is presented, and solutions to the issues of speckle noise and safety are presented.     

High‐resolution autostereoscopic 3‐D projection display with a space‐dividing iris‐plane shutter

Abstract— A high‐resolution autostereoscopic 3‐D projection display with a polarization‐control space dividing the iris‐plane liquid‐crystal shutter is proposed. The polarization‐control iris‐plane shutter can control the direction of stereo images without reducing the image quality of the microdis‐play. This autostereoscopic 3‐D projection display is 2‐D/3‐D switchable and has a high resolution and high luminance. In addition, it has no cross‐talk between the left and right viewing zones, a simple structure, and the capability to show multi‐view images.     

4032 ppi High‐resolution OLED microdisplay

A 0.5‐inch Ultra Extended Graphics Array (UXGA) organic light‐emitting diodes microdisplay has been developed with 6.3 μm pixel pitch. Not only 4032 ppi high resolution but high frame rate, low power consumption, wide viewing angle, and high luminance have been achieved. This newly developed organic light‐emitting diodes microdisplay is suitable for Near‐to‐Eye display applications, especially electronic viewfinders.     

Projection‐type integral 3‐D display with distortion compensation

Abstract— Our research is aimed at developing a spatial‐imaging‐type integral three‐dimensional (3‐D) display based on an integral photography method using an extremely high‐resolution projector. One problem with the projection‐type integral 3‐D display is that geometrical distortion in projected elemental images causes spatial deformation of the displayed 3‐D image. In this study, a general relationship between the geometric distortion of elemental images and the spatial deformation of reconstructed 3‐D images were analyzed. A projection‐type integral 3‐D display with a distortion compensator which corrects the geometrical distortions of projected images in real‐time have been developed. The deformation of the displayed 3‐D images was significantly reduced by the distortion compensation, and the displayed 3‐D images had a resolution of 182 (H) × 140 (V) pixels and a viewing angle of 24.5°.     

A high‐resolution autostereoscopic display system with a wide viewing angle using an LCOS projector array

Abstract— This study develops an autostereoscopic display based on a multiple miniature projector array to provide a scalable solution for a high‐resolution 3‐D display with large viewing freedom. To minimize distortion and dispersion, and to maximize the modulation transfer function (MTF) of the projection image to optimize 3‐D image quality, a dedicated projection lens and an accurate six‐axis adjusting platform for the miniature projector were designed and fabricated. Image‐blending technology based on a lookup table was adopted to combine images from 30 miniature projectors into a seamless single image. The result was a 35‐in. autostereoscopic display with 60 views ata 30° viewing angle, 90° FOV, and large range of viewing distance. The proposed system exhibits very smooth motion parallax when viewers move around in front of it.     

Laser display with single‐mirror MEMS scanner

Abstract— While arrayed DMD and LCD microdisplays are well‐established approaches for visualization tasks, image‐forming laser scanners are an emerging technology used to build miniaturized projection displays. A directly modulated RGB‐laser module consisting of diode lasers for red and blue and a frequency‐doubled semiconductor laser for green with color combining optics form the light source for the laser scanner have been developed. Subsequent beam‐shaping optics suppresses unwanted stray light and enables optimum illumination of the scanning mirror. The MEMS device features a single scanning mirror oscillating in two directions in resonant mode. This requires appropriate data delivery realized by a custom‐made driving logic, which converts the pixel stream originally arranged in rows and columns to the Lissajous‐like spot trajectory on the screen. Additionally, the increased image brightness at the vertical and horizontal borders of the field of view (FOV) is also compensated by the modulation of laser power. Theoretical investigations of the resulting maximum achievable system transmission are presented. Different systems, such as an extremely miniaturized monochrome projection head with an integrated diode laser and a full‐color projector have been realized. Important problems to be tackled are fast analog modulation of the laser power with high resolution and improved suppression of stray light and speckle.     

High‐visibility 2‐D/3‐D LCD with HDDP arrangement and its optical characterization methods

Abstract— A 3.1‐in.‐diagonal 2‐D/3‐D LCD with a novel pixel arrangement, called horizontally double‐density pixels (HDDP), for high‐quality 3‐D images has been developed. 3‐D visibility has been improved by broadening the qualified stereoscopic viewing space (QSVS) where high‐quality 3‐D images can be seen. In order to evaluate the QSVS, optical characterization methods, based on the ergonomics for stereoscopy, such as 3‐D crosstalk, interocular luminance difference and 3‐D moiré have been proposed. The implementation results show that these methods can correctly evaluate high‐visibility autostereoscopic displays.     

Projection‐based head‐tracking 3‐D displays

Abstract— This paper describes the construction and operation of four 3‐D displays in which each display produces two images for each eye and thus fits into the category of projection‐based binocular stereoscopic displays. The four 3‐D displays described are pico‐projector‐based, liquid‐ crystal—on—silicon (LCOS) conventional projector‐based, 120‐Hz digital‐light‐processor (DLP) projector‐ based, and the HELIUM3D system. In the first three displays, images are produced on a direct‐view LCD whose conventional backlight is replaced with a projection illumination source that is controlled by a multi‐user head tracker; novel steering optics direct the projector output to regions referred to as exit pupils located at the viewers' eyes. In the HELIUM3D display, the image information is supplied by a horizontally scanned, fast, light valve whose output is controlled by a spatial light modulator (SLM) to direct images to the appropriate viewers' eyes. The current statu s and the multimodal potential of the HELIUM3D display are described.     

Ultra‐high‐resolution 1058‐ppi OLED displays with 2.78‐in size using CAAC‐IGZO FETs with tandem OLED device and single OLED device

We fabricated new 2.78‐in 1058‐ppi organic light‐emitting diode (OLED) displays. The displays used OLED devices with a tandem structure and a single structure and a field effect transistor (FET) using c‐axis aligned crystalline In–Ga–Zn–O (CAAC‐IGZO) for an active layer and employing the 1.5‐µm rule over a glass substrate. Even in the displays with such high resolution exceeding 1000 ppi, crosstalk that was observed in the lower luminance region was suppressed. The displays achieved high color reproducibility and reduced viewing angle dependence.     

Toward high‐resolution,inkjet‐printed,quantum dot light‐emitting diodes for next‐generation displays

We have investigated the possibility of fabricating quantum dot light‐emitting diodes (QLEDs) using inkjet printing technology, which is the most attractive method for the full‐color patterning of QLED displays. By controlling the quantum dot (QD) ink formulation and inkjet printing condition, we successfully patterned QLED pixels in the 60‐in ultrahigh definition TV format, which has a resolution of 73 pixels per inch. The inkjet‐printed QLEDs exhibited a maximum luminance of 2500 cd/m². Although the performance of inkjet‐printed QLEDs is low compared with that of QLEDs fabricated using the spin‐coating process, our results clearly indicate that the inkjet printing technology is suitable for patterning QD emissive layers to realize high‐resolution, full‐color QLED displays.     

Pixel structures using the negative feedback method for high‐resolution active matrix organic light‐emitting diode displays

Novel two pixel structures are proposed for high‐resolution active matrix organic light‐emitting diode displays. The proposed two pixels (pixel structures A and B) use the negative feedback method for high‐resolution displays that requires to have small‐sized storage capacitance. The proposed pixel structures A and B improve the luminance uniformity by reducing the voltage distortion in the storage capacitor. However, the proposed pixel structure A is vulnerable to the organic light‐emitting diode (OLED) degradation because the anode voltage of the OLED affects the emission current. In order to compensate the OLED degradation, the proposed pixel structure B stores the turn‐on voltage of OLEDs in the storage capacitor. The simulation results show that the emission current error of the proposed pixel structure B is improved by four times in comparison with the proposed pixel structure A when the OLED turn‐on voltage increases by 0.1 V. Also, the emission current error of the proposed pixel structure B when the threshold voltage of driving thin‐film transistors varies from ?2.2 to ?1.8 V is from ?0.69 least significant bit (LSB) to 0.13 LSB, which shows the excellent luminance uniformity. The proposed pixels are designed for 5.5‐in. full high‐definition displays.     

Glare‐limited appearances in HDR images

Abstract— Intraocular glare and simultaneous contrast control appearance in high‐dynamic‐range (HDR) images. This paper describes unique test targets that simulate real images. These targets change the HDR range by 500 times, without significantly changing the veiling glare on the retina. As well, these targets have nearly constant simultaneous contrast. The range of appearances possible from HDR images with different average luminances were measured. The targets displayed a maximum luminance range of 5.4 log units. Using magnitude estimates (MagEst) of appearances, the relationship between luminance and lightness from white to black was measured. With one exception, only small changes in appearance with large changes in dynamic range were found. It was also found that appearance was scene‐dependent. The same dark grays (MagEst = 10) were observed with luminances of 10, 4.2, 1.1, and 0.063, depending on the percentage of white area in the surround. Glare from more white increases the retinal luminance of the test areas. Simultaneous contrast counteracts glare by making the appearance range (white‐black) with a much smaller range of luminances. Appearance is controlled by both the optical scattered light and the spatial processing. A single tone‐scale function of luminance cannot describe appearance controlled by scatter and spatial processing.     

A 470 × 235‐ppi poly‐Si TFT‐LCD for high‐resolution 2‐D and 3‐D autostereoscopic displays

Abstract— We have developed a 470 × 235‐ppi poly‐Si TFT‐LCD with a novel pixel arrangement, called HDDP (horizontally double‐density pixels), for high‐resolution 2‐D and 3‐D autostereoscopic displays. 3‐D image quality is especially high in a lenticular‐lens‐equipped 3‐D mode because both the horizontal and vertical resolutions are high, and because these resolutions are equal. 3‐D and 2‐D images can be displayed simultaneously in the same picture. In addition, 3‐D images can be displayed anywhere and 2‐D characters can be made to appear at different depths with perfect legibility. No switching of 2‐D/3‐D modes is necessary, and the design's thin and uncomplicated structure makes it especially suitable for mobile terminals.     

High‐quality direct‐view display combining multiple integral 3D images

This paper proposes a method for combining multiple integral three‐dimensional (3D) images using direct‐view displays to obtain high‐quality results. A multi‐image combining optical system (MICOS) is used to enlarge and combine multiple integral 3D images without gaps. An optical design with a simple lens configuration that does not require a diffuser plate prevents the deterioration in resolution resulting from lens arrangement errors and the diffuser plate. An experiment was performed to compare a previously developed method with the proposed method, and the latter showed a significant improvement in image quality. A method for expanding the effective viewing angle of the proposed optical design was also developed, and its effectiveness was confirmed experimentally. A prototype device of the proposed optical design was constructed using a high‐density organic light‐emitting diode (OLED) panel with 8K resolution and 1058 ppi pixel density to achieve 311 (H) × 175 (V) elemental images, a viewing angle of 20.6° in both the horizontal and vertical directions, and a display size of 9.1 in. In addition, the proposed optical design enabled making device considerably thinner, ie, with a thickness of only 47 mm.     

OLED‐based pico‐projection system

Abstract— Two pico‐projection systems, a monochrome green and a full‐color system, based on high‐efficiency OLED microdisplays (VGA; pixel size, 12 μm) are presented. Both optical systems are described by a numerical aperture of about 0.3, a magnification of 15x, and a working distance of 300–360 mm. The frequency limit of both systems is 42 cycles/mm at an image contrast of about 60%. The monochrome projection system with a volume smaller than 10 cm³ consists of one green OLED and a projection lens with five elements. The measured luminance in the image plane is about 0.061 lm. The image has a diagonal of 150 mm with a working distance of about 300 mm and has a considerable image contrast of 396:1. The second system combines three high‐brightness OLEDs, red, green, and blue colored, together with a projection lens and an image‐combining element, and an X‐Cube to achieve full‐color projection. The estimated luminance value for the three‐panel projection unit with an OLED luminance of 10,000 cd/m² for each display will be about Φcalculated = 0.147 lm. In this paper, the system concepts, the optical designs, and the realized prototypes of the monochrome and full‐color projection system are presented.     

Advanced level‐adaptive overdrive (ALAO) method applicable to full‐HD LC TVs

Abstract— In this paper, we report on the development of an advanced level‐adaptive overdrive (ALAO) method applicable to full‐HD LC TVs, which makes it possible not only to reduce the gray‐level response time to less than one fourth, but also to improve the S/N ratio for still images by 10 dB; and further to reduce the circuit cost for the LAO method to almost half that of the prior LAO circuit. We have applied the ALAO method to a full‐HD LC front projector with 1.84 Mpixels and a screen size larger than 100 in., and we obtained high‐quality full‐HD moving images.     

Color reproduction with a various number of sub‐pixels

Monochrome reflective‐type displays are widely used for portable reading applications such as electric papers because this type does not need a back light unit and can be used outdoors for a long time. Color reflective‐type displays without back light units are desired to expand the market further. The current color reproduction is based on three sub‐pixel red, green and blue (RGB) methods, and when used in reflective type, its luminance is reduced to a third of that of monochrome type. Adding a white sub‐pixel to the current method can improve the luminance, making the sub‐pixel number four. However, in the case of a high resolution display with a four‐sub‐pixel method, the pixel structure is complex, and the luminance improvement may be limited. Instead of increasing the sub‐pixel number, two sub‐pixel methods are investigated. These methods can improve luminance with limited color gamut. The performances are compared with those of other methods quantitatively.     

Characterization of electronic displays using CMOS single‐photon avalanche diode image sensors

Advanced complementary metal‐oxide semiconductor‐compatible single‐photon avalanche diode array technology is progressing rapidly and is being deployed in a wide range of applications. We report for the first time the use of a complementary metal‐oxide semiconductor‐compatible single‐photon avalanche diode array to perform detailed optical measurements on pixels of an organic light‐emitting diode microdisplay at very high sampling rate, very low light level, and over a very wide dynamic range of luminance. This offers a clear demonstration of the huge potential of this single‐photon avalanche diode technology to reveal hitherto obscure details of the optical characteristics of individual and groups of organic light‐emitting diode pixels.     

Color conversion for a multi‐primary display using linear interpolation on equi‐luminance plane method (LIQUID)

Abstract— A color‐conversion method, which converts an XYZ tristimulus value to a set of display signals for a multi‐primary display, has been developed. It is necessary for a display‐device driver in a color‐management system to select an optimal set of display signals that presents a given colorimetric value on a display screen. A multi‐primary display, which has more than four primaries, has more than two sets of display signals that generate the same colorimetric value theoretically. A display‐device driver, therefore, has to be designed according to a certain rule in order to choose a unique set of display signals for one colorimetric value only. To meet the requirement, a linear interpolation method, which uses three points on an equi‐luminance plane in XYZ tristimulus value space, has been developed. The developed method assigns three points on an equi‐luminance plane in XYZ tristimulus value space to ensure smoothness of a continuous gradation of display signals. The colorimetric accuracy of the developed method was evaluated with a six‐primary DLP? projector. Colorimetric error in CIELAB space between the targets and reproductions is about 0.8 on average and 2.9 at maximum.