Projector placement planning for high quality visualizations on real-world colored objects

Many visualization applications benefit from displaying content on real-world objects rather than on a traditional display (e.g., a monitor). This type of visualization display is achieved by projecting precisely controlled illumination from multiple projectors onto the real-world colored objects. For such a task, the placement of the projectors is critical in assuring that the desired visualization is possible. Using ad hoc projector placement may cause some appearances to suffer from color shifting due to insufficient projector light radiance being exposed onto the physical surface. This leads to an incorrect appearance and ultimately to a false and potentially misleading visualization. In this paper, we present a framework to discover the optimal position and orientation of the projectors for such projection-based visualization displays. An optimal projector placement should be able to achieve the desired visualization with minimal projector light radiance. When determining optimal projector placement, object visibility, surface reflectance properties, and projector-surface distance and orientation need to be considered. We first formalize a theory for appearance editing image formation and construct a constrained linear system of equations that express when a desired novel appearance or visualization is possible given a geometric and surface reflectance model of the physical surface. Then, we show how to apply this constrained system in an adaptive search to efficiently discover the optimal projector placement which achieves the desired appearance. Constraints can be imposed on the maximum radiance allowed by the projectors and the projectors' placement to support specific goals of various visualization applications. We perform several real-world and simulated appearance edits and visualizations to demonstrate the improvement obtained by our discovered projector placement over ad hoc projector placement.     

Exploiting DLP Illumination Dithering for Reconstruction and Photography of High-Speed Scenes

In this work, we recover fast moving scenes by exploiting the high-speed illumination “dithering” of cheap and easily available digital light processing (DLP) projectors. We first show how to reverse-engineer the temporal dithering for off-the-shelf projectors, using a high-speed camera. DLP dithering can produce temporal patterns commonly used in active vision techniques. Since the dithering occurs at a very high frame-rate, such illumination-based methods can be “speed up” for fast scenes. We demonstrate this with three applications, each of which only requires a single slide to be displayed by the DLP projector. The quality of the result is determined by the camera frame-rate available to the user. Pairing a high-speed camera and a DLP projector, we demonstrate structured light reconstruction at 100 Hz. With the same camera and three or more DLP projectors, we show photometric stereo and demultiplexing applications at 300 Hz. Finally, with a real-time (60 Hz) or still camera, we show that DLP illumination acts as a very fast flash, allowing strobe photography of high-speed scenes. We discuss, in depth, some characteristics of the temporal dithering with a case study of a particular projector. Finally, we describe limitations, trade-offs and other issues relating to this work.     

Practical Radiometric Compensation for Projection Display on Textured Surfaces using a Multidimensional Model

Radiometric compensation methods remove the effect of the underlying spatially varying surface reflectance of the texture when projecting on textured surfaces. All prior work sample the surface reflectance dependent radiometric transfer function from the projector to the camera at every pixel that requires the camera to observe tens or hundreds of images projected by the projector. In this paper, we cast the radiometric compensation problem as a sampling and reconstruction of multi‐dimensional radiometric transfer function that models the color transfer function from the projector to an observing camera and the surface reflectance in a unified manner. Such a multi‐dimensional representation makes no assumption about linearity of the projector to camera color transfer function and can therefore handle projectors with non‐linear color transfer functions(e.g. DLP, LCOS, LED‐based or laser‐based). We show that with a well‐curated sampling of this multi‐dimensional function, achieved by exploiting the following key properties, is adequate for its accurate representation: (a) the spectral reflectance of most real‐world materials are smooth and can be well‐represented using a lower‐dimension function; (b) the reflectance properties of the underlying texture have strong redundancies – for example, multiple pixels or even regions can have similar surface reflectance; (c) the color transfer function from the projector to camera have strong input coherence. The proposed sampling allows us to reduce the number of projected images that needs to be observed by a camera by up to two orders of magnitude, the minimum being only two. We then present a new multi‐dimensional scattered data interpolation technique to reconstruct the radiometric transfer function at a high spatial density (i.e. at every pixel) to compute the compensation image. We show that the accuracy of our interpolation technique is higher than any existing methods.     

Surround structured lighting: 3-D scanning with orthographic illumination

This paper presents a new system for rapidly acquiring complete 3-D surface models using a single orthographic structured light projector, a pair of planar mirrors, and one or more synchronized cameras. Using the mirrors, we project structured light patterns that illuminate the object from all sides (not just the side of the projector) and are able to observe the object from several vantage points simultaneously. This system requires that projected planes of light to be parallel, so we construct an orthographic projector using a Fresnel lens and a commercial DLP projector. A single Gray code sequence is used to encode a set of vertically-spaced light planes within the scanning volume, and five views of the illuminated object are obtained from a single image of the planar mirrors located behind it. From each real and virtual camera we recover a dense 3-D point cloud spanning the entire object surface using traditional structured light algorithms. A key benefit of this design is to ensure that each point on the object surface can be assigned an unambiguous Gray code sequence, despite the possibility of being illuminated from multiple directions. In addition to presenting a prototype implementation, we also develop a complete set of mechanical alignment and calibration procedures for utilizing orthographic projectors in computer vision applications. As we demonstrate, the proposed system overcomes a major hurdle to achieving full 360° reconstructions using a single structured light sequence by eliminating the need for merging multiple scans or multiplexing several projectors.     

Turning a Digital Camera into an Absolute 2D Tele‐Colorimeter

We present a simple and effective technique for absolute colorimetric camera characterization, invariant to changes in exposure/aperture and scene irradiance, suitable in a wide range of applications including image‐based reflectance measurements, spectral pre‐filtering and spectral upsampling for rendering, to improve colour accuracy in high dynamic range imaging. Our method requires a limited number of acquisitions, an off‐the‐shelf target and a commonly available projector, used as a controllable light source, other than the reflected radiance to be known. The characterized camera can be effectively used as a 2D tele‐colorimeter, providing the user with an accurate estimate of the distribution of luminance and chromaticity in a scene, without requiring explicit knowledge of the incident lighting power spectra. We validate the approach by comparing our estimated absolute tristimulus values (XYZ data in ) with the measurements of a professional 2D tele‐colorimeter, for a set of scenes with complex geometry, spatially varying reflectance and light sources with very different spectral power distribution.     

The illumination-invariant matching of deterministic localstructure in color images

The availability of multiple spectral measurements at each pixel in an image provides important additional information for recognition. Spectral information is of particular importance for applications where spatial information is limited. Such applications include the recognition of small objects or the recognition of small features on partially occluded objects. We introduce a feature matrix representation for deterministic local structure in color images. Although feature matrices are useful for recognition, this representation depends on the spectral properties of the scene illumination. Using a linear model for surface spectral reflectance with the same number of parameters as the number of color bands, we show that changes in the spectral content of the illumination correspond to linear transformations of the feature matrices, and that image plane rotations correspond to circular shifts of the matrices. From these relationships, we derive an algorithm for the recognition of local surface structure which is invariant to these scene transformations. We demonstrate the algorithm with a series of experiments on images of real objects     

DLP和LCD投影机对比分析

投影机是一种重要的计算机图形图像显示设备。本文从介绍投影机的分类入手,分别详细介绍分析了LCD和DLP两种主流投影机的工作原理,并且就这两种投影机的技术特点进行了性能对比分析。可以看出,数字化的DLP投影技术在许多方面是具有很大优势的。     

基于阵列相机的多光谱成像系统光谱重建算法

针对传统三通道RGB相机在光源光谱已知条件下不能完全恢复物体表面光谱反射率的缺点,本文构造一套多光谱成像阵列相机系统。该阵列相机采用12个大恒DH-HV1300FM型相机,且11个镜头装有波长不同的滤光片。本文结合阵列相机多通道数的优势,提出一种MSIS-GOC（Multi-spectral Imaging System based on Group of Camera）算法,能够可靠并有效地重建场景的光谱反射率。仿真实验结果分析验证了该系统的有效性。     

Colorimetric characterization of LCD and DLP projection displays

Abstract— Successful color‐management of projection systems depends on knowledge of their characteristics. In this study, two typical portable projectors were characterized. The projectors are based on different technologies, liquid‐crystal display (LCD) and digital light‐processing (DLP). Measurements were made with a spectroradiometer. The properties measured were spectral characteristics and the intensity of the primary and white colors, basic colorimetric characteristics, inter‐channel dependency, tone characteristics, color‐tracking characteristics, spatial non‐uniformity, dependency on background, and temporal stability. Based on the characterization results, the possibility of color‐management of the tested projectors is discussed.     

Agile Spectrum Imaging: Programmable Wavelength Modulation for Cameras and Projectors

We advocate the use of quickly‐adjustable, computer‐controlled color spectra in photography, lighting and displays. We present an optical relay system that allows mechanical or electronic color spectrum control and use it to modify a conventional camera and projector. We use a diffraction grating to disperse the rays into different colors, and introduce a mask (or LCD/DMD) in the optical path to modulate the spectrum. We analyze the trade‐offs and limitations of this design, and demonstrate its use in a camera, projector and light source. We propose applications such as adaptive color primaries, metamer detection, scene contrast enhancement, photographing fluorescent objects, and high dynamic range photography using spectrum modulation.     

Spatially uniform colors for projectors and tiled displays

Abstract— A major issue when setting up multi‐projector tiled displays is the spatial non‐uniformity of the color throughout the display's area. Indeed, the chromatic properties do not only vary between two different projectors, but also between different spatial locations inside the displaying area of one single projector. A new method for calibrating the colors of a tiled display is presented. An iterative algorithm to construct a correction table which makes the luminance uniform over the projected area of one single projector is presented first. This so‐called intra‐projector calibration uses a standard camera as a luminance measuring device and can be processed in parallel for all projectors. Once the color inside each projector is spatially uniform, the set of displayable colors — the color gamut — of each projector is measured. On the basis of these measurements, the goal of the inter‐projector calibration is to find an optimal gamut shared by all the projectors. Finding the optimal color gamut displayable by n projectors in time O(n) is shown, and the color conversion from one specific color gamut to the common global gamut is derived. The method of testing it on a tiled display consisting of 48 projectors with large chrominance shifts was experimentally validated.     

Color reflectance modeling using a polychromatic laser range sensor

A system for simultaneously measuring the 3-D shape and color properties of objects is described. Range data are obtained by triangulation over large volumes of the scene, whereas color components are separated by means of a white laser. Details are given concerning the modeling and the calibration of the system for bidirectional reflectance-distribution functions measurements. A reflection model is used to interpret the data collected with the system in terms of the underlying physical properties of the target. These properties are the diffuse reflectance of the body material, the Fresnel reflectance of the air media interface, and the slope surface roughness of the interface. Experimental results are presented for the extraction of these parameters. By allowing the subtraction of highlights from color images and the compensation for surface orientation, spectral reflectance modeling can help to understand 3-D scenes. A practical example is given where a color and range image is processed to yield uniform regions according to material pigmentation     

Color‐tuning projection system for adaptive combination of high brightness and wide color gamut

Abstract— The demand for projectors with high brightness and wide color gamut has been increasing; however, UHP lamp projectors cannot deliver those two qualities efficiently and simultaneously because of its color‐separation system. The newly developed projection system — “Color‐Tuning Projection System” — realizes the adaptive combination of high brightness and wide color gamut with one projector. This projector features a fourth liquid‐crystal panel — “Color Tuner” — with a 3LCD optical engine, which controls yellow light separately from the RGB light of a UHP lamp. This color‐tuner‐based optical engine — “Color‐Tuning Optical Engine” — and a new color‐conversion signal‐processing algorithm — “Adaptive Color Conversion Algorithm” — controls the yellow‐light volume and corrects color‐shifted pixels according to the brightness and chromaticity analysis of the input image, key technologies of the Color‐Tuning Projection System. This additional panel system enables the projector to ach ieve up to 115% higher brightness and 120% wider color gamut according to the input image. This paper presents an innovative design concept, a novel technology regarding brightness and a color‐gamut conversion projection system, and the characteristics of the prototype.     

Color nonuniformity in projection-based displays: analysis and solutions

Large-area displays made up of several projectors show significant variation in color. Here, we identify different projector parameters that cause the color variation and study their effects on the luminance and chrominance characteristics of the display. This work leads to the realization that luminance varies significantly within and across projectors, while chrominance variation is relatively small, especially across projectors of same model. To address this situation, we present a method to achieve luminance matching across all pixels of a multiprojector display that results in photometrically uniform displays. We use a camera as a measurement device for this purpose. Our method comprises a one-time calibration step that generates a per channel per projector luminance attenuation map (LAM), which is then used to correct any image projected on the display at interactive rates on commodity graphics hardware. To the best of our knowledge, this is the first effort to match luminance across all the pixels of a multiprojector display.     

Multifocal projection: a multiprojector technique for increasing focal depth

In this paper, we describe a novel multifocal projection concept that applies conventional video projectors and camera feedback. Multiple projectors with differently adjusted focal planes, but overlapping image areas are used. They can be either differently positioned in the environment or can be integrated into a single projection unit. The defocus created on an arbitrary surface is estimated automatically for each projector pixel. If this is known, a final image with minimal defocus can be composed in real-time from individual pixel contributions of all projectors. Our technique is independent of the surfaces' geometry, color and texture, the environment light, as well as of the projectors' position, orientation, luminance, and chrominance.     

Two LCOS full color projector with efficient LED illumination engine

LED-based projectors have numerous advantages compared to traditional projectors, but their light output is limited by the limited brightness of the LEDs. With an efficient illumination engine design we can build an LED projector with a moderate light output and with superior properties. In this paper we present a compact LED projector with two ‘Liquid Crystal On Silicon’ panels and four LEDs (R, 2 × G, B). We use two panels instead of the classical three panels and will still have the same performance and moreover a reduced volume and cost. The illumination system consists of a custom made monolithic component (GTLP) that is combining many functions. We have also integrated some methods to increase the brightness of the LEDs by pulsing them. Additional methods, such as using an extra PBS to combine both color channels, are implemented to increase the contrast. After investigating the promising simulation results (119 lm D65 light with very high contrast and uniformity), we built a demonstrator setup. Our demonstrator produces a moderate light output (37.3 lm) on screen with a sufficient contrast ratio and a very good uniformity. In spite of semi color sequential working, the color breakup and crosstalk are negligible. The difference in performance and possible improvements will be discussed in the paper.     

High‐efficiency optical system for ultra‐short‐throw‐distance projector based on multi‐laser light source

Abstract— A novel laser‐light‐source projector having the three outstanding features of high brightness, ultra‐short throw distance, and high color reproduction has been developed.These features have recently come to be required in the high‐end projector market. The technologies for the laser‐light‐source projectors fully utilize the advantages of lasers, such as high luminance, small étendue, and high color purity. By integrating a triple‐rod illumination system with a multi‐laser light source and an ultra‐wide‐angle projection system, the developed high‐efficiency optical system has achieved a brightness of 7000 lm and a throw ratio of 0.28 with an image size of 100–150 in. Another new technology, laser color processing (LCP), has offered vivid color reproduction which has a color gamut that is up to 180% wider than the BT.709 standard without appearing unnaturally colored. Furthermore, a speckle suppression effect produced by the multi‐laser light source has been demonstrated. In this paper, an overview of these newly developed technologies that are used in the novel laser‐light‐source projector is presented, and solutions to the issues of speckle noise and safety are presented.     

基于HDR的投影机亮度曲线标定

多台投影机的无缝拼接校正过程主要是投影机的几何校正和颜色校正。多台投影机的颜色校正要求是求出每台投影机的颜色曲线,并对其颜色的相同输出做对应的输入使之达到全局的颜色统一。采用基于相机HDR的方法,先求出相机的亮度响应曲线,再以相机的亮度响应曲线为基础,使相机的输入为投影机的输出,间接求得投影机的亮度响应曲线。实验结果表明,基于HDR的亮度曲线测量,与直接使用照度计比较,结果相差在10%以内。     

Seamless ultrathin rear projection display

A new architecture for a thin (2‐cm depth) rear projection display is described. In order to achieve this small depth, a very high density of rear projectors is used. Three prototype displays using rear projectors on both 5‐ and 2‐cm pitch arrays are described. The displays can achieve an effective screen pixel pitch of as small as 0.5 mm, which makes this technology competitive in terms of resolution with fine pitch LED displays; however, orders of magnitude fewer LEDs are required: Each rear projector requires only one white LED and a color liquid crystal light modulator. In the three prototypes, the projector light modulators utilize 101‐cm (40 in.), 80‐cm (31.5 in.), and 60‐cm (24 in.) diagonal liquid crystal display glass. To minimize cost, no lenses are utilized for the rear projectors. An RGB LED array may augment the projector array, which provides a low resolution component of the image onto which the high resolution component is superimposed by the projector array. Edge gaps between active areas on adjacent LCD glass units are completely eliminated by the rear projection approach enabling low profile wall‐size seamless displays. Display contrast depends on rear projection screen design.     

Color and brightness appearance issues in tiled displays

Large-format displays created by tiling multiple, projected images have been used for decades in flight simulators and entertainment and are commercially available in a variety of forms. More recently, various research organizations have built custom display walls out of commodity projectors to support research in visualization, large-format display, and interaction. In these settings, making the display appear as a single, seamless surface has proven challenging. Where tiles overlap, they create bright seams. The tiles vary in color and brightness, not only from tile to tile, but within each tile. Each projector has a slightly different color gamut, caused by variations in the bulb, color filters, and digital processing (contrast, brightness, and gamma) for the projector. The spatial variation in brightness has two causes. First, the light from a projection system doesn't uniformly illuminate the screen. Second, the light doesn't scatter uniformly out of the front of the screen, making the perceived brightness depend on the viewing angle. In some projectors, the projected light's color also varies across the tile's face, resulting in unwanted tints in the images. I describe what causes these variations and what can be done about them