Automatic Registration of Multi‐Projector Domes Using a Single Uncalibrated Camera

In this paper we present a novel technique for easily calibrating multiple casually aligned projectors on spherical domes using a single uncalibrated camera. Using the prior knowledge of the display surface being a dome, we can estimate the camera intrinsic and extrinsic parameters and the projector to display surface correspondences automatically using a set of images. These images include the image of the dome itself and a projected pattern from each projector. Using these correspondences we can register images from the multiple projectors on the dome. Further, we can register displays which are not entirely visible in a single camera view using multiple pan and tilted views of an uncalibrated camera making our method suitable for displays of different size and resolution. We can register images from any arbitrary viewpoint making it appropriate for a single head‐tracked user in a 3D visualization system. Also, we can use several cartographic mapping techniques to register images in a manner that is appropriate for multi‐user visualization. Domes are known to produce a tremendous sense of immersion and presence in visualization systems. Yet, till date, there exists no easy way to register multiple projectors on a dome to create a high‐resolution realistic visualizations. To the best of our knowledge, this is the first method that can achieve accurate geometric registration of multiple projectors on a dome simply and automatically using a single uncalibrated camera.     

Scalable Multi‐view Registration for Multi‐Projector Displays on Vertically Extruded Surfaces

Recent work have shown that it is possible to register multiple projectors on non‐planar surfaces using a single uncalibrated camera instead of a calibrated stereo pair when dealing with a special class of non‐planar surfaces, vertically extruded surfaces. However, this requires the camera view to contain the entire display surface. This is often an impossible scenario for large displays, especially common in visualization, edutainment, training and simulation applications. In this paper we present a new method that can achieve an accurate geometric registration even when the field‐of‐view of the uncalibrated camera can cover only a part of the vertically extruded display at a time. We pan and tilt the camera from a single point and employ a multi‐view approach to register the projectors on the display. This allows the method to scale easily both in terms of camera resolution and display size. To the best of our knowledge, our method is the first to achieve a scalable multi‐view geometric registration of large vertically extruded displays with a single uncalibrated camera. This method can also handle a different situation of having multiple similarly oriented cameras in different locations, if the camera focal length is known.     

Advances towards high‐resolution pack‐and‐go displays: A survey

Abstract— Tiled displays provide high resolution and large scale simultaneously. Projectors can project on any available surface. Thus, it is possible to create a large high‐resolution display by simply tiling multiple projectors on any available regular surface. The tremendous advancement in projection technology has made projectors portable and affordable. One can envision displays made of multiple such projectors that can be packed in one's car trunk, carried from one location to another, deployed at each location easily to create a seamless high‐resolution display, and, finally, dismantled in minutes to be taken to the next location — essentially a pack‐and‐go display. Several challenges must be overcome in order to realize such pack‐and‐go displays. These include allowing for imperfect uncalibrated devices, uneven non‐diffused display surfaces, and a layman user via complete automation in deployment that requires no user invention. We described the advances we have made in addressing these challenges for the most common case of planar display surfaces. First, we present a technique to allow imperfect projectors. Next, we present a technique to allow a photometrically uncalibrated camera. Finally, we present a novel distributed architecture that renders critical display capabilities such as self‐calibration, scalability, and reconfigurability without any user intervention. These advances are important milestones towards the development of easy‐to‐use multi‐projector displays that can be deployed anywhere and by anyone.     

Registration techniques for using imperfect and partially calibrated devices in planar multi-projector displays

Multi-projector displays today are automatically registered, both geometrically and photometrically, using cameras. Existing registration techniques assume pre-calibrated projectors and cameras that are devoid of imperfections such as lens distortion. In practice, however, these devices are usually imperfect and uncalibrated. Registration of each of these devices is often more challenging than the multi-projector display registration itself. To make tiled projection-based displays accessible to a layman user we should allow the use of uncalibrated inexpensive devices that are prone to imperfections. In this paper, we make two important advances in this direction. First, we present a new geometric registration technique that can achieve geometric alignment {\em in the presence of severe projector lens distortion} using a relatively inexpensive low-resolution camera. This is achieved via a closed-form model that relates the projectors to cameras, in planar multi-projector displays, using rational Bezier patches. This enables us to geometrically calibrate a 3000 x 2500 resolution planar multi-projector display made of 3 x 3 array of nine severely distorted projectors using a low resolution (640 x 480) VGA camera. Second, we present a photometric self-calibration technique for a projector-camera pair. This allows us to photometrically calibrate the same display made of nine projectors using a photometrically uncalibrated camera. To the best of our knowledge, this is the first work that allows geometrically imperfect projectors and photometrically uncalibrated cameras in calibrating multi-projector displays.     

Easy Calibration of a Multi-projector Display System

In this paper, we present a method for the geometric calibration of a multi-projector display system. The method is such that in order to calibrate the system, the user is only required to place the projectors and capture a single image of the images projected from them onto a planar screen using a hand-held camera. The problem to be solved is divided into the image registration for stitching different projector images into a single seamless image and the image rectification for making the image have the correct rectangular shape. The proposed method is characterized by simultaneously solving both of them from only a single image, which makes the calibration procedures easy. The method assumes an uncalibrated camera and partially calibrated projectors in which only focal lengths are unknown among the internal parameters. In the paper, we first prove the uniqueness of solutions to the problem, which was unclear in the previous studies, and then present a stable numerical algorithm for actually finding the solution. We present several experimental results for synthetic data, in which we show the relation between the calibration accuracy and several factors, and also present experimental results for real data, in which we demonstrate that the proposed method can calibrate a real system with sufficient accuracy for a number of layouts of the projectors.     

Asynchronous distributed calibration for scalable and reconfigurable multi-projector displays

Centralized techniques have been used until now when automatically calibrating (both geometrically and photometrically) large high-resolution displays created by tiling multiple projectors in a 2D array. A centralized server managed all the projectors and also the camera(s) used to calibrate the display. In this paper, we propose an asynchronous distributed calibration methodology via a display unit called the plug-and-play projector (PPP). The PPP consists of a projector, camera, computation and communication unit, thus creating a self-sufficient module that enables an asynchronous distributed architecture for multi-projector displays. We present a single-program-multiple-data (SPMD) calibration algorithm that runs on each PPP and achieves a truly scalable and reconfigurable display without any input from the user. It instruments novel capabilities like adding/removing PPPs from the display dynamically, detecting faults, and reshaping the display to a reasonable rectangular shape to react to the addition/removal/faults. To the best of our knowledge, this is the first attempt to realize a completely asynchronous and distributed calibration architecture and methodology for multi-projector displays.     

多投影面沉浸式虚拟环境系统的颜色校正

提出一种颜色校正的函数逼近优化模型来解决多投影面沉浸式虚拟环境系统的颜色校正问题。该颜色校正模型，采用数码相机作为颜色反馈的测量仪器，获得一个基准投影面的亮度和色度转换函数；为其他投影面分别寻找一个亮度修正函数和色度修正函数，使得各投影面与基准投影面的亮度和色度转换函数之间的L2距离最小，进而根据各修正函数来补偿各投影机的输入响应差别以达到它们之间的颜色一致。给出了逼近校正方法的算法及其程序实现，通过一个三通道的试验系统，证明了此方法对解决多投影面沉浸式虚拟环境系统颜色校正问题的有效性。     

Robust and accurate visual echo cancellation in a full-duplex projector-camera system

In this paper we study the problem of "visual echo" in a full-duplex projector-camera system for tele-collaboration applications. Visual echo is defined as the appearance of projected contents observed by the camera. It can potentially saturate the projected contents, similar to audio echo in telephone conversation. Our approach to visual echo cancelation includes an off-line calibration procedure that records the geometric and photometric transfer between the projector and the camera in a look-up table. During run-time, projected contents in the captured video are identified using the calibration information and suppressed, therefore achieving the goal of canceling visual echo. Our approach can accurately handle full color images under arbitrary reflectance of display surfaces and photometric response of the projector or camera. It is robust to geometric registration errors and quantization effect, therefore particularly effective for high-frequency contents such as texts and hand drawings. We demonstrate the effectiveness of our approach with a variety of real images in a full-duplex projector-camera system.     

Camera-based detection and removal of shadows from interactive multiprojector displays

Front-projection displays are a cost-effective and increasingly popular method for large format visualization and immersive rendering of virtual models. New approaches to projector tiling, automatic calibration, and color balancing have made multiprojector display systems feasible without undue infrastructure changes and maintenance. As a result, front-projection displays are being used to generate seamless, visually immersive worlds for virtual reality and visualization applications with reasonable cost and maintenance overhead. However, these systems suffer from a fundamental problem: Users and other objects in the environment can easily and inadvertently block projectors, creating shadows on the displayed image. Shadows occlude potentially important information and detract from the sense of presence an immersive display may have conveyed. We introduce a technique that detects and corrects shadows in a multiprojector display while it is in use. Cameras observe the display and compare observations with an expected image to detect shadowed regions. These regions are transformed to the appropriate projector frames, where corresponding pixel values are increased and/or attenuated. In display regions where more than one projector contributes to the image, shadow regions are eliminated.     

Multi-frame estimation of planar motion

Traditional plane alignment techniques are typically performed between pairs of frames. We present a method for extending existing two-frame planar motion estimation techniques into a simultaneous multi-frame estimation, by exploiting multi-frame subspace constraints of planar surfaces. The paper has three main contributions: 1) we show that when the camera calibration does not change, the collection of all parametric image motions of a planar surface in the scene across multiple frames is embedded in a low dimensional linear subspace; 2) we show that the relative image motion of multiple planar surfaces across multiple frames is embedded in a yet lower dimensional linear subspace, even with varying camera calibration; and 3) we show how these multi-frame constraints can be incorporated into simultaneous multi-frame estimation of planar motion, without explicitly recovering any 3D information, or camera calibration. The resulting multi-frame estimation process is more constrained than the individual two-frame estimations, leading to more accurate alignment, even when applied to small image regions.     

A scalable distributed paradigm for multi-user interaction with tiled rear projection display walls

We present the first distributed paradigm for multiple users to interact simultaneously with large tiled rear projection display walls. Unlike earlier works, our paradigm allows easy scalability across different applications, interaction modalities, displays and users. The novelty of the design lies in its distributed nature allowing well-compartmented, application independent, and application specific modules. This enables adapting to different 2D applications and interaction modalities easily by changing a few application specific modules. We demonstrate four challenging 2D applications on a nine projector display to demonstrate the application scalability of our method: map visualization, virtual graffiti, virtual bulletin board and an emergency management system. We demonstrate the scalability of our method to multiple interaction modalities by showing both gesture-based and laser-based user interfaces. Finally, we improve earlier distributed methods to register multiple projectors. Previous works need multiple patterns to identify the neighbors, the configuration of the display and the registration across multiple projectors in logarithmic time with respect to the number of projectors in the display. We propose a new approach that achieves this using a single pattern based on specially augmented QR codes in constant time. Further, previous distributed registration algorithms are prone to large misregistrations. We propose a novel radially cascading geometric registration technique that yields significantly better accuracy. Thus, our improvements allow a significantly more efficient and accurate technique for distributed self-registration of multi-projector display walls.     

基于多投影的多视点自动立体三维显示系统

多视点自动立体显示有望成为今后主流的三维显示技术,它是一种无需借助任何辅助观察设备的多视点、多观察区、高分辨率、显示效果逼真的三维显示方式。阐述了基于多投影的多视点自动立体显示系统的设计原理,详细地描述了系统的软硬件构架,建立了基于多投影仪和水平光学各向异性显示结构的自动立体显示样机,开发了投影仪阵列自动校准系统,提高了投影仪的校准精度,避免了因投影仪数目多而导致的繁琐的校准过程。实验结果能够给观众带来逼真的三维视觉体验。     

CAVE for collaborative patient room design: analysis with end-user opinion contrasting method

Several studies indicate that virtual reality (VR) systems are useful for end-user participation in an environmental design process. However, these systems can be costly and thus support for the decision whether to invest in a VR of some type is useful. This study presents a novel method for analysing the usefulness of a VR system for the purpose of end-user participation. We collected qualitative end-user opinion data in the real environment and then contrasted this data with the capabilities of a VR system. Additionally, to better understand the capabilities of the VR used, we examined how the end-users perceive the used virtual environment, which in this case was CAVE, an immersive VR system where projectors are directed to the walls of a room-sized cube. In this way, we analysed whether the same functions and elements identified by end-users on the actual wards could also be evaluated in the CAVE. Eleven nurses and 11 patients participated in the study by evaluating a bathroom and/or four patient rooms modelled by the CAVE and the actual hospital wards. The CAVE was convenient for evaluating most issues identified by the study participants in the actual hospital wards, i.e. aesthetics; correct location of equipment, supplies and materials; distraction by or the good companion of other patients as well as window position and size and living/workspace. However, it was not possible to evaluate with full certainty the possibilities for bracing against grab bars or other objects in the VR, and this was found to be relevant to the independent functioning of patients with limited mobility. Also, due to the relatively low luminance levels of projectors, evaluations regarding lighting were considered unreliable. Moreover, end-users were not always certain about the sizes and sufficiency of space in the CAVE. Solutions to overcome these limitations were proposed.     

CAVE立体显示系统的搭建及立体图像的几何校正

为了解决由投影机的位置不准确而引起的立体图像视差畸变问题,根据立体图像产生方式以及显示方式的不同,给出了适合于工程上搭建"经济型"CAVE立体显示系统的几种实现方式,包括主动立体方式、主动变被动方式以及直接被动立体方式.将帧缓存中双眼的视景图像分别拷贝到纹理内存中,通过重投影变换计算出B样条曲面控制点坐标并绘制B样条曲面,分别映射纹理内存中双眼图像的纹理到各自的B样条曲面上.通过对左右眼立体图像分别进行几何校正,实现了CAVE多通道立体显示的无缝拼接;通过微调 B样条曲面的控制点坐标以及确定曲面阶数,实现了左右立体图像局部位置的调整以及图像平滑度的调整.实验结果显示,该方法不影响视景实时绘制.     

Toward the light field display: autostereoscopic rendering via a cluster of projectors

Ultimately, a display device should be capable of reproducing the visual effects observed in reality. In this paper we introduce an autostereoscopic display that uses a scalable array of digital light projectors and a projection screen augmented with microlenses to simulate a light field for a given three-dimensional scene. Physical objects emit or reflect light in all directions to create a light field that can be approximated by the light field display. The display can simultaneously provide many viewers from different viewpoints a stereoscopic effect without head tracking or special viewing glasses. This work focuses on two important technical problems related to the light field display; calibration and rendering. We present a solution to automatically calibrate the light field display using a camera and introduce two efficient algorithms to render the special multi-view images by exploiting their spatial coherence. The effectiveness of our approach is demonstrated with a four-projector prototype that can display dynamic imagery with full parallax.     

Film and television industry cloud exhibition design based on 3D imaging and virtual reality

The three-dimensional virtual scene can provide users with a visual three-dimensional virtual environment, with various multimedia channels such as sound, video, force feedback equipment, etc., to bring users a completely immersive interactive experience. This paper introduces 3D imaging and virtual reality technology in the film and television industry cloud exhibition, and develops a virtual display platform. First of all, this paper divides the registration into two processes: camera calibration and joint calibration of the camera and laser scanner based on the calibration results. Camera calibration can determine the plane model of the calibration board in the camera coordinate system, and the joint calibration uses the RANSAC algorithm to extract the point cloud of the plane model of the calibration board, and then optimizes the distance between the points in the plane model point cloud and the corresponding plane in the camera coordinate system Find the optimal transformation between the two sets of data, and then calculate the registration relationship between the point cloud and the image. Secondly, this article conducts a demand analysis of the film and television industry cloud exhibition platform based on virtual reality technology, including the business goals set by the platform, platform system analysis, overall design, and system operating environment and configuration requirements. This model provides a feasible solution for the visual interaction of the cloud exhibition design of the film and television industry.     

Color nonuniformity in projection-based displays: analysis and solutions

Large-area displays made up of several projectors show significant variation in color. Here, we identify different projector parameters that cause the color variation and study their effects on the luminance and chrominance characteristics of the display. This work leads to the realization that luminance varies significantly within and across projectors, while chrominance variation is relatively small, especially across projectors of same model. To address this situation, we present a method to achieve luminance matching across all pixels of a multiprojector display that results in photometrically uniform displays. We use a camera as a measurement device for this purpose. Our method comprises a one-time calibration step that generates a per channel per projector luminance attenuation map (LAM), which is then used to correct any image projected on the display at interactive rates on commodity graphics hardware. To the best of our knowledge, this is the first effort to match luminance across all the pixels of a multiprojector display.     

Viewing geometric protein structures from inside a CAVE

We have developed general modeling software for a Cave Automatic Virtual Environment (CAVE); one of its applications is modeling 3D protein structures, generating both outside-in and inside-out views of geometric models. An advantage of the CAVE over other virtual environments is that multiple viewers can observe the same scene at the same time and place. Our software is scalable-from high-end virtual environments such as the CAVE, to mid-range immersive desktop systems, down to low-end graphics workstations. In the current configuration, a parallel Silicon Graphics Power Challenge supercomputer architecture performs the computationally intensive construction of surface patches remotely, and sends the results through the I-WAY (Information Wide Area Year) using VBNS (Very-high-Bandwidth Network Systems) to the graphics machines that drive the CAVE and our graphics visualization software, Valvis (Virtual ALpha shapes VISualizer)     

自适应复杂环境的投影图像校正算法

为实现投影仪在任意表面上的自适应投影,提出一种可以在日常环境光照条件下进行自动几何校正和色彩补偿的投影图像校正算法.该算法使用基于二进制编码结构光的离散映射集合方法,通过计算对应像素映射关系消除投影图像的几何畸变;然后应用一种优化的朗伯特反射模型构造投影表面的纹理空间,对原始投影图像进行预处理和全局亮度补偿,最终实现投影图像的颜色补偿.实验结果表明,基于文中算法在复杂环境中进行投影,可以在不规则几何表面上投影出符合视觉期望的无变形图像,并能够有效地消除投影表面固有纹理和环境光照对投影图像的干扰;将自适应复杂环境的投影图像校正算法应用于传统投影机摆脱了投影机对投影幕布的依赖,可以在任意环境中实现近似于白色平板幕布的投影效果.