Autocalibration of multiprojector CAVE-like immersive environments

In this paper, we present the first method for the geometric autocalibration of multiple projectors on a set of CAVE-like immersive display surfaces including truncated domes and 4 or 5-wall CAVEs (three side walls, floor, and/or ceiling). All such surfaces can be categorized as swept surfaces and multiple projectors can be registered on them using a single uncalibrated camera without using any physical markers on the surface. Our method can also handle nonlinear distortion in the projectors, common in compact setups where a short throw lens is mounted on each projector. Further, when the whole swept surface is not visible from a single camera view, we can register the projectors using multiple pan and tilted views of the same camera. Thus, our method scales well with different size and resolution of the display. Since we recover the 3D shape of the display, we can achieve registration that is correct from any arbitrary viewpoint appropriate for head-tracked single-user virtual reality systems. We can also achieve wallpapered registration, more appropriate for multiuser collaborative explorations. Though much more immersive than common surfaces like planes and cylinders, general swept surfaces are used today only for niche display environments. Even the more popular 4 or 5-wall CAVE is treated as a piecewise planar surface for calibration purposes and hence projectors are not allowed to be overlapped across the corners. Our method opens up the possibility of using such swept surfaces to create more immersive VR systems without compromising the simplicity of having a completely automatic calibration technique. Such calibration allows completely arbitrary positioning of the projectors in a 5-wall CAVE, without respecting the corners.     

Multifocal projection: a multiprojector technique for increasing focal depth

In this paper, we describe a novel multifocal projection concept that applies conventional video projectors and camera feedback. Multiple projectors with differently adjusted focal planes, but overlapping image areas are used. They can be either differently positioned in the environment or can be integrated into a single projection unit. The defocus created on an arbitrary surface is estimated automatically for each projector pixel. If this is known, a final image with minimal defocus can be composed in real-time from individual pixel contributions of all projectors. Our technique is independent of the surfaces' geometry, color and texture, the environment light, as well as of the projectors' position, orientation, luminance, and chrominance.     

Exploiting DLP Illumination Dithering for Reconstruction and Photography of High-Speed Scenes

In this work, we recover fast moving scenes by exploiting the high-speed illumination “dithering” of cheap and easily available digital light processing (DLP) projectors. We first show how to reverse-engineer the temporal dithering for off-the-shelf projectors, using a high-speed camera. DLP dithering can produce temporal patterns commonly used in active vision techniques. Since the dithering occurs at a very high frame-rate, such illumination-based methods can be “speed up” for fast scenes. We demonstrate this with three applications, each of which only requires a single slide to be displayed by the DLP projector. The quality of the result is determined by the camera frame-rate available to the user. Pairing a high-speed camera and a DLP projector, we demonstrate structured light reconstruction at 100 Hz. With the same camera and three or more DLP projectors, we show photometric stereo and demultiplexing applications at 300 Hz. Finally, with a real-time (60 Hz) or still camera, we show that DLP illumination acts as a very fast flash, allowing strobe photography of high-speed scenes. We discuss, in depth, some characteristics of the temporal dithering with a case study of a particular projector. Finally, we describe limitations, trade-offs and other issues relating to this work.     

一种简易的线结构光投影三维形状获取方法

利用普通的LCD投影仪和数码相机获得物体的三维形状.投影仪作为光源,投射线结构光在物体表面,从数码相机拍摄的照片序列中,恢复物体的形状.提出了根据已经标定好的相机参数,继续对投影仪进行标定的方法;同时,针对物体表面颜色/纹理较为复杂的情况,提出了一次投射三色线结构光的改进方法,克服了不易恢复此类物体形状的困难.实验结果表明,该方法可以获得物体的稠密点云用于进一步的重建,适于组建一个廉价但却保持相对精度的扫描系统.     

Structured light self-calibration with vanishing points

This paper introduces the vanishing points to self-calibrate a structured light system. The vanishing points permit to automatically remove the projector’s keystone effect and then to self-calibrate the projector–camera system. The calibration object is a simple planar surface such as a white paper. Complex patterns and 3D calibrated objects are not required any more. The technique is compared to classic calibration and validated with experimental results.     

Projector placement planning for high quality visualizations on real-world colored objects

Many visualization applications benefit from displaying content on real-world objects rather than on a traditional display (e.g., a monitor). This type of visualization display is achieved by projecting precisely controlled illumination from multiple projectors onto the real-world colored objects. For such a task, the placement of the projectors is critical in assuring that the desired visualization is possible. Using ad hoc projector placement may cause some appearances to suffer from color shifting due to insufficient projector light radiance being exposed onto the physical surface. This leads to an incorrect appearance and ultimately to a false and potentially misleading visualization. In this paper, we present a framework to discover the optimal position and orientation of the projectors for such projection-based visualization displays. An optimal projector placement should be able to achieve the desired visualization with minimal projector light radiance. When determining optimal projector placement, object visibility, surface reflectance properties, and projector-surface distance and orientation need to be considered. We first formalize a theory for appearance editing image formation and construct a constrained linear system of equations that express when a desired novel appearance or visualization is possible given a geometric and surface reflectance model of the physical surface. Then, we show how to apply this constrained system in an adaptive search to efficiently discover the optimal projector placement which achieves the desired appearance. Constraints can be imposed on the maximum radiance allowed by the projectors and the projectors' placement to support specific goals of various visualization applications. We perform several real-world and simulated appearance edits and visualizations to demonstrate the improvement obtained by our discovered projector placement over ad hoc projector placement.     

基于微机电系统振镜的多频相移正弦结构光系统相位展开误差分析

随着增强现实和虚拟现实技术的发展,以及元宇宙概念的兴起,三维重建技术作为重要的内容获取手段得到了广泛的应用。其中,结构光重建技术以其高精度且不受物体表面材质纹理影响的特点,得到了研究者的重点关注。传统的结构光三维重建主要使用基于数字光处理的投影仪来投影编码图案,然而,数字光处理投影仪存在尺寸大、功率高、价格昂贵等缺点,限制了其在众多应用中的便利性。因此,微机电系统振镜因其体积小、成本低、帧率高等特点,在越来越多的三维扫描系统中,被考虑作为投影仪投影编码结构光。但限于只能投影单向条纹图案和存在红外激光器散斑效应带来的噪声的缺点,传统的基于三角测量方法并不适用。因此,该文采用相位-高度模型,搭建了基于微机电系统振镜的三维扫描系统。针对散斑效应引起的噪声,本文通过实验对比了基于微机电系统振镜常用的 3 种时间相位解包裹算法的抗噪性能。结果表明,多频层级法和负指数拟合法的抗噪性能较好,精度较高,而多频外差法的抗噪性能较差。该研究结果可为研究人员选择解相方法提供参考。     

Automatic Registration of Multi‐Projector Domes Using a Single Uncalibrated Camera

In this paper we present a novel technique for easily calibrating multiple casually aligned projectors on spherical domes using a single uncalibrated camera. Using the prior knowledge of the display surface being a dome, we can estimate the camera intrinsic and extrinsic parameters and the projector to display surface correspondences automatically using a set of images. These images include the image of the dome itself and a projected pattern from each projector. Using these correspondences we can register images from the multiple projectors on the dome. Further, we can register displays which are not entirely visible in a single camera view using multiple pan and tilted views of an uncalibrated camera making our method suitable for displays of different size and resolution. We can register images from any arbitrary viewpoint making it appropriate for a single head‐tracked user in a 3D visualization system. Also, we can use several cartographic mapping techniques to register images in a manner that is appropriate for multi‐user visualization. Domes are known to produce a tremendous sense of immersion and presence in visualization systems. Yet, till date, there exists no easy way to register multiple projectors on a dome to create a high‐resolution realistic visualizations. To the best of our knowledge, this is the first method that can achieve accurate geometric registration of multiple projectors on a dome simply and automatically using a single uncalibrated camera.     

Auto‐calibration of a projector–camera stereo system for projection mapping

Standard camera and projector calibration techniques use a checkerboard that is manually shown at different poses to determine the calibration parameters. Furthermore, when image geometric correction must be performed on a three‐dimensional (3D) surface, such as projection mapping, the surface geometry must be determined. Camera calibration and 3D surface estimation can be costly, error prone, and time‐consuming when performed manually. To address this issue, we use an auto‐calibration technique that projects a series of Gray code structured light patterns. These patterns are captured by the camera to build a dense pixel correspondence between the projector and camera, which are used to calibrate the stereo system using an objective function, which embeds the calibration parameters together with the undistorted points. Minimization is carried out by a greedy algorithm that minimizes the cost at each iteration with respect to both calibration parameters and noisy image points. We test the auto‐calibration on different scenes and show that the results closely match a manual calibration of the system. We show that this technique can be used to build a 3D model of the scene, which in turn with the dense pixel correspondence can be used for geometric screen correction on any arbitrary surface.     

Autocalibration of a projector-camera system

This paper presents a method for calibrating a projector-camera system that consists of multiple projectors (or multiple poses of a single projector), a camera, and a planar screen. We consider the problem of estimating the homography between the screen and the image plane of the camera or the screen-camera homography, in the case where there is no prior knowledge regarding the screen surface that enables the direct computation of the homography. It is assumed that the pose of each projector is unknown while its internal geometry is known. Subsequently, it is shown that the screen-camera homography can be determined from only the images projected by the projectors and then obtained by the camera, up to a transformation with four degrees of freedom. This transformation corresponds to arbitrariness in choosing a two-dimensional coordinate system on the screen surface and when this coordinate system is chosen in some manner, the screen-camera homography as well as the unknown poses of the projectors can be uniquely determined. A noniterative algorithm is presented, which computes the homography from three or more images. Several experimental results on synthetic as well as real images are shown to demonstrate the effectiveness of the method.     

基于平面的便携式结构光系统柔性标定方法

对于便携式结构光系统, 该系统必须适合方便使用. 因此, 针对该系统的标定方法也必须采用方便且便宜的设备, 诸如具有两个或三个正交平面或者需要额外固定措施的一些设备则不宜采用. 同时针对快速三维获取的目的, 应该采用估计便携式结构光系统投影矩阵的方法. 本文针对便携式结构光系统的上述应用需求, 提出了一种基于平面的便携式结构光系统柔性标定方法. 该方法需要一个标定模板和一个参考模板. 标定模板被固定在一个平面上, 而参考模板则通过LCD投影仪被投射到该平面上. 通过交比和极几何约束, 可以获得分别对投影仪和相机的世界坐标系和图像坐标系的点对应坐标. 通过这些点对应坐标, 即可完成整个系统的标定. 实验结果表明该方法具有很高的准确性和鲁棒性．     

基于误差校正的高精度三维测量系统

基于投影仪的结构光三维测量系统已得到了广泛的应用,但由于投影仪与相机的非线性特点,使用现有相位恢复算法得到的相位值具有一定的误差,影响了三维测量的精度。提出了一种基于二次型误差校正的高精度相位恢复算法。通过对标准平板进行相位恢复,分析其误差分布特征;用二次多项式对该误差进行逼近,得到误差的分布规律;根据求出的二次型误差模型对投影仪结构光系统得到的初始相位值进行修正,得到了高精度的相位恢复结果。实验表明,该算法比现有算法具有更高的相位恢复和三维测量精度。     

Development of a parameterization image stitching algorithm for ultrashort throw laser MEMS projectors

We present a novel design of a parameterization image stitching algorithm for ultrashort throw laser MEMS projectors. The resultant method allows the use of projectors with short or even long throw ratios to achieve ultrashort throw projection through a parameterized algorithm to stitch multiple images into one single frame, alleviating the trade‐off between the resolution and frame rate in conventional laser MEMS displays. To evaluate the effectiveness, we construct a test system that consists of three off‐the‐shelf projectors with a throw ratio of 1.3 and three planar reflective mirrors. Using our method, the integrated system yields an ultrashort throw ratio of 0.26, greatly reducing the projection distance from 12 to 2.6 ft for the 120‐in. screen in diagonal. Moreover, our algorithm can be fine‐tuned to correct for image distortion, intensity variation, and edge mismatch induced by two‐axis light beam steering, thereby further improving the image quality.     

Calibrating a Structured Light Stripe System: A Novel Approach

The problem associated with calibrating a structured light stripe system is that known world points on the calibration target do not normally fall onto every light stripe plane illuminated from the projector. We present in this paper a novel calibration method that employs the invariance of the cross ratio to overcome this problem. Using 4 known non-coplanar sets of 3 collinear world points and with no prior knowledge of the perspective projection matrix of the camera, we show that world points lying on each light stripe plane can be computed. Furthermore, by incorporating the homography between the light stripe and image planes, the 4 × 3 image-to-world transformation matrix for each stripe plane can also be recovered. The experiments conducted suggest that this novel calibration method is robust, economical, and is applicable to many dense shape reconstruction tasks.     

Camera–projector matching using unstructured video

This paper presents a novel approach for matching 2-D points between a video projector and a digital camera. Our method is motivated by camera–projector applications for which the projected image needs to be warped to prevent geometric distortion. Since the warping process often needs geometric information on the 3-D scene obtained from a triangulation, we propose a technique for matching points in the projector to points in the camera based on arbitrary video sequences. The novelty of our method lies in the fact that it does not require the use of pre-designed structured light patterns as is usually the case. The backbone of our application lies in a function that matches activity patterns instead of colors. This makes our method robust to pose, severe photometric and geometric distortions. It also does not require calibration of the color response curve of the camera–projector system. We present quantitative and qualitative results with synthetic and real-life examples, and compare the proposed method with the scale invariant feature transform (SIFT) method and with a state-of-the-art structured light technique. We show that our method performs almost as well as structured light methods and significantly outperforms SIFT when the contrast of the video captured by the camera is degraded.     

Registration techniques for using imperfect and partially calibrated devices in planar multi-projector displays

Multi-projector displays today are automatically registered, both geometrically and photometrically, using cameras. Existing registration techniques assume pre-calibrated projectors and cameras that are devoid of imperfections such as lens distortion. In practice, however, these devices are usually imperfect and uncalibrated. Registration of each of these devices is often more challenging than the multi-projector display registration itself. To make tiled projection-based displays accessible to a layman user we should allow the use of uncalibrated inexpensive devices that are prone to imperfections. In this paper, we make two important advances in this direction. First, we present a new geometric registration technique that can achieve geometric alignment {\em in the presence of severe projector lens distortion} using a relatively inexpensive low-resolution camera. This is achieved via a closed-form model that relates the projectors to cameras, in planar multi-projector displays, using rational Bezier patches. This enables us to geometrically calibrate a 3000 x 2500 resolution planar multi-projector display made of 3 x 3 array of nine severely distorted projectors using a low resolution (640 x 480) VGA camera. Second, we present a photometric self-calibration technique for a projector-camera pair. This allows us to photometrically calibrate the same display made of nine projectors using a photometrically uncalibrated camera. To the best of our knowledge, this is the first work that allows geometrically imperfect projectors and photometrically uncalibrated cameras in calibrating multi-projector displays.     

面向交叉点网络的符号解码方法的研究

计算机视觉是近年来研究的热点,结构光视觉是双目计算机视觉的一个分支,该技术采用一台投射器代替双目立体视觉的一台摄像机,由于其投射的结构光具有编码唯一性的特点,从而解决了双目视觉中像素匹配的难题。结构光技术中解码成为了一项重要的工作,根据符号的特点提出一种基于交叉点网络的符号解码方法。该方法有效地实现了交叉点的提取,并形成全局的交叉点网络。针对细化的十字交叉变形,研究了交叉点合并方法,提出了基于识圈法的内部交叉点定位方法,以及基于向量角度的符号分类方法,有效地实现了符号码字的还原。最终实现物体的三维重建目标。     

Parametric study using distributed computing and genetic algorithm in TFT‐LCD projector

We present an analysis of the parameters affecting light engines in liquid crystal display (LCD) projectors, using actual LCD projectors as examples. The significant parameters affecting light engines in LCD projectors are evaluated using the light uniformity as the determining standard. Optimization is performed on those parameters that affect the uniformity as well as the rendering effects of the projectors, by employing our own distributed genetic algorithm. The rotation angle of the collimator and its position are shown to improve the uniformity of the LCD projection, but we find that their effect on improving the brightness is not significant. Increased length of a projectors' light tube promotes better integration results. In our study, we effectively improved the performance index of an LCD projector, increasing projection uniformity by 4% (to 91.5%) and the brightness by 12% (to 7.9 lm).     

Easy Calibration of a Multi-projector Display System

In this paper, we present a method for the geometric calibration of a multi-projector display system. The method is such that in order to calibrate the system, the user is only required to place the projectors and capture a single image of the images projected from them onto a planar screen using a hand-held camera. The problem to be solved is divided into the image registration for stitching different projector images into a single seamless image and the image rectification for making the image have the correct rectangular shape. The proposed method is characterized by simultaneously solving both of them from only a single image, which makes the calibration procedures easy. The method assumes an uncalibrated camera and partially calibrated projectors in which only focal lengths are unknown among the internal parameters. In the paper, we first prove the uniqueness of solutions to the problem, which was unclear in the previous studies, and then present a stable numerical algorithm for actually finding the solution. We present several experimental results for synthetic data, in which we show the relation between the calibration accuracy and several factors, and also present experimental results for real data, in which we demonstrate that the proposed method can calibrate a real system with sufficient accuracy for a number of layouts of the projectors.     

一种基于条纹扫描的结构光三维信息获取方法*

提出一种基于结构光的三维信息获取方法,利用投影仪向物体投射黑色条纹,移动的条纹扫描整个物体,根据条纹在物体上的变形来估计物体的三维形状,实现了三维信息的获取。该方法仅使用投影仪和数码相机等简单设备,实现简单快速,精度相对比较高,适用范围广,可扩展性强。