Projection‐based head‐tracking 3‐D displays

Abstract— This paper describes the construction and operation of four 3‐D displays in which each display produces two images for each eye and thus fits into the category of projection‐based binocular stereoscopic displays. The four 3‐D displays described are pico‐projector‐based, liquid‐ crystal—on—silicon (LCOS) conventional projector‐based, 120‐Hz digital‐light‐processor (DLP) projector‐ based, and the HELIUM3D system. In the first three displays, images are produced on a direct‐view LCD whose conventional backlight is replaced with a projection illumination source that is controlled by a multi‐user head tracker; novel steering optics direct the projector output to regions referred to as exit pupils located at the viewers' eyes. In the HELIUM3D display, the image information is supplied by a horizontally scanned, fast, light valve whose output is controlled by a spatial light modulator (SLM) to direct images to the appropriate viewers' eyes. The current statu s and the multimodal potential of the HELIUM3D display are described.     

基于单投影仪与柱面反射镜的沉浸感显示系统

在分析传统沉浸感显示系统优缺点的基础上,设计并实现了一种新型的沉浸感显示系统。该系统使用柱面反射镜对单投影仪投射光线进行反射,并在弧形背投幕上成像,能够获得无缝画面,实现了广角度虚拟场景的连贯显示。通过合理设计柱面反射镜形状,实现了投影画面在水平方向上的均匀放大。通过对图像进行预变形可以基本消除因投影幕曲率而产生的投影画面形变,同时采用背投技术使参与者在虚拟场景中活动更加自如。该系统克服了传统的多投影仪或多显示器沉浸感显示系统中存在的图像拼接问题,且易于构建,经实验验证能够获得良好的沉浸感显示效果。     

Non‐planar projection performance evaluation and projector pose optimization

In projector applications, the actual display effect is primarily dependent on two aspects: projector performance parameters (intrinsic properties), and projector placement (extrinsic properties). Many studies have been conducted on the impact of the original resolution, illumination, and contrast ratio on projection performance. However, few studies have focused on the influence of extrinsic properties, including the position and orientation of the projector on the projection effect. In this paper, three projection performance evaluation criteria—projection resolution, resolution difference, and projection distance difference—are proposed in terms of extrinsic properties. Based on these evaluation criteria, a projection performance evaluation function was constructed. Through this function, as well as optimization employing the multi‐start and Monte Carlo methods, projector pose parameters corresponding to optimal projection effects can be obtained. Evaluations demonstrated that the proposed projection performance evaluation criteria could correctly describe the impacts of projector placement on projection effects. Projection systems arranged according to the projector pose optimization methods achieved good performance on screens in difference shapes with different numbers of projectors, where the resolution and display improved with no change of projector hardware parameters. The optimal projector poses calculated using these optimization methods can provide theoretical guidance for practical projector placement.     

Image correction based on homography in stereoscopic projection display

Abstract— Stereoscopic and autostereoscopic projection‐display systems use projector arrays to present stereoscopic images, and each projector casts one parallax image of a stereoscopic scene. Because of the position shift of the projectors, the parallax images have geometric deformation, which influences the quality of the displayed stereoscopic images. In order to solve this problem, a method based on homography is proposed. The parallax images are pre‐transformed before they are projected, and then the stereoscopic images without geometric distortion can be obtained. An autostereoscopic projection‐display system is developed to present the images with and without calibration. Experimental results show that this method works effectively.     

Super‐short‐focus front projector with aspheric‐mirror projection optical system

Abstract— We have developed a new front projector with a super‐short projection distance of 0.65 m at 100 in. We installed a newly developed reflective‐type projection optical system in this projector, which was composed of only four aspheric mirrors that were free of color aberrations. Using this new system and a single‐chip DMD?, we projected a picture that had excellent sharpness and high contrast from 40 to 100 in. This paper describes the principles, design, and characteristics of the new WT600? front projector.     

Vivid image projection system using e‐Paper active screen

Projected image on a screen is not always vivid enough when it is projected in a bright room. We generally have to choose either vivid image in a dark room or dull image in a bright room. We have suggested a new projection system which can realize high contrast image projection in a bright room. It is consisted of electronic paper screen, projector, and light‐emitting diode room light. The electronic paper (e‐Paper) screen alternates its whole surface white/black with 120 Hz. Room lights are controlled with the same frequency of 120 Hz with the opposite phase as that of the e‐Paper screen. We have confirmed that our new system has achieved higher contrast than the conventional projection system and also enough readability of paper documents in the room simultaneously. We have thus confirmed that our new system can bring an ideal condition, for audience, which can realize a vivid image projection in a bright room.     

High‐efficiency optical system for ultra‐short‐throw‐distance projector based on multi‐laser light source

Abstract— A novel laser‐light‐source projector having the three outstanding features of high brightness, ultra‐short throw distance, and high color reproduction has been developed.These features have recently come to be required in the high‐end projector market. The technologies for the laser‐light‐source projectors fully utilize the advantages of lasers, such as high luminance, small étendue, and high color purity. By integrating a triple‐rod illumination system with a multi‐laser light source and an ultra‐wide‐angle projection system, the developed high‐efficiency optical system has achieved a brightness of 7000 lm and a throw ratio of 0.28 with an image size of 100–150 in. Another new technology, laser color processing (LCP), has offered vivid color reproduction which has a color gamut that is up to 180% wider than the BT.709 standard without appearing unnaturally colored. Furthermore, a speckle suppression effect produced by the multi‐laser light source has been demonstrated. In this paper, an overview of these newly developed technologies that are used in the novel laser‐light‐source projector is presented, and solutions to the issues of speckle noise and safety are presented.     

Auto‐calibration of a projector–camera stereo system for projection mapping

Standard camera and projector calibration techniques use a checkerboard that is manually shown at different poses to determine the calibration parameters. Furthermore, when image geometric correction must be performed on a three‐dimensional (3D) surface, such as projection mapping, the surface geometry must be determined. Camera calibration and 3D surface estimation can be costly, error prone, and time‐consuming when performed manually. To address this issue, we use an auto‐calibration technique that projects a series of Gray code structured light patterns. These patterns are captured by the camera to build a dense pixel correspondence between the projector and camera, which are used to calibrate the stereo system using an objective function, which embeds the calibration parameters together with the undistorted points. Minimization is carried out by a greedy algorithm that minimizes the cost at each iteration with respect to both calibration parameters and noisy image points. We test the auto‐calibration on different scenes and show that the results closely match a manual calibration of the system. We show that this technique can be used to build a 3D model of the scene, which in turn with the dense pixel correspondence can be used for geometric screen correction on any arbitrary surface.     

注意力机制的曲面沉浸式投影系统补偿

目的 沉浸式投影系统已广泛运用于虚拟现实系统之中,然而沉浸式投影系统中的互反射现象严重影响着虚拟现实系统的落地使用。沉浸式投影系统的互反射是指由于投影机光线和屏幕反射光线相互叠加造成的亮度冗余现象,严重影响了投影系统的成像质量和人眼的视觉感受。为此,本文提出一种新的基于互反射通道(inter-reflection channel,IRC)先验和注意力机制的神经网络。方法 IRC先验基于这样一个事实,即大多数受到互反射影响的投影图像都包含一些亮度较高的区域。高亮度区域往往受互反射影响更为严重,而低亮度区域受互反射影响程度较低。根据这一规律,采用IRC先验作为注意力图的监督样本,获取补偿图像的亮度区域信息。同时,为了对投影图像不同区域按影响程度进行差异化补偿,提出一种新的由两个相同子网络构成的补偿网络结构Pair-Net。结果 实验对比了4种现有方法,Pair-Net在ROI(region of interesting)指标分析上取得了明显优势,在人眼感受上有显著的效果提升。结论 本文提出的基于注意力机制的网络模型能够针对不同区域进行差异化补偿,很大程度上消除了互反射影响,提升了沉浸式投影系统的成像质量。     

The Visual Computing of Projector‐Camera Systems

This article focuses on real‐time image correction techniques that enable projector‐camera systems to display images onto screens that are not optimized for projections, such as geometrically complex, coloured and textured surfaces. It reviews hardware‐accelerated methods like pixel‐precise geometric warping, radiometric compensation, multi‐focal projection and the correction of general light modulation effects. Online and offline calibration as well as invisible coding methods are explained. Novel attempts in super‐resolution, high‐dynamic range and high‐speed projection are discussed. These techniques open a variety of new applications for projection displays. Some of them will also be presented in this report.     

Seamless ultrathin rear projection display

A new architecture for a thin (2‐cm depth) rear projection display is described. In order to achieve this small depth, a very high density of rear projectors is used. Three prototype displays using rear projectors on both 5‐ and 2‐cm pitch arrays are described. The displays can achieve an effective screen pixel pitch of as small as 0.5 mm, which makes this technology competitive in terms of resolution with fine pitch LED displays; however, orders of magnitude fewer LEDs are required: Each rear projector requires only one white LED and a color liquid crystal light modulator. In the three prototypes, the projector light modulators utilize 101‐cm (40 in.), 80‐cm (31.5 in.), and 60‐cm (24 in.) diagonal liquid crystal display glass. To minimize cost, no lenses are utilized for the rear projectors. An RGB LED array may augment the projector array, which provides a low resolution component of the image onto which the high resolution component is superimposed by the projector array. Edge gaps between active areas on adjacent LCD glass units are completely eliminated by the rear projection approach enabling low profile wall‐size seamless displays. Display contrast depends on rear projection screen design.     

Spatially uniform colors for projectors and tiled displays

Abstract— A major issue when setting up multi‐projector tiled displays is the spatial non‐uniformity of the color throughout the display's area. Indeed, the chromatic properties do not only vary between two different projectors, but also between different spatial locations inside the displaying area of one single projector. A new method for calibrating the colors of a tiled display is presented. An iterative algorithm to construct a correction table which makes the luminance uniform over the projected area of one single projector is presented first. This so‐called intra‐projector calibration uses a standard camera as a luminance measuring device and can be processed in parallel for all projectors. Once the color inside each projector is spatially uniform, the set of displayable colors — the color gamut — of each projector is measured. On the basis of these measurements, the goal of the inter‐projector calibration is to find an optimal gamut shared by all the projectors. Finding the optimal color gamut displayable by n projectors in time O(n) is shown, and the color conversion from one specific color gamut to the common global gamut is derived. The method of testing it on a tiled display consisting of 48 projectors with large chrominance shifts was experimentally validated.     

A 65‐in. slim (255‐mm depth) laser TV with wide‐angle projection optical system

Abstract— A high‐definition laser TV that employs a newly developed laser light source and a super‐wide‐angle projection optical system has been developed. This adoption of a laser light source with three primary colors helped to achieve an extremely wide color gamut, and, in addition, a compact optical engine, which has been optimized to the laser light source and contributed to the achievement of the stylish design of a large screen of 65 in., with the depth being only 255 mm.     

基于投影二维平面靶标的投影仪现场标定方法研究

投影仪的标定是结构光三维视觉测量系统应用的基础。提出了一种利用投影二维平面靶标对投影仪进行现场标定的新方法。与传统的投影仪标定相比,该方法不需要精密的标定设备,提高了标定精度,简化了标定过程,能够满足现场不断变换的要求。实验证明:该标定方法获得的结果较准确,方法简便易行,能够满足现场不断变化的测量需求。     

Development of a parameterization image stitching algorithm for ultrashort throw laser MEMS projectors

We present a novel design of a parameterization image stitching algorithm for ultrashort throw laser MEMS projectors. The resultant method allows the use of projectors with short or even long throw ratios to achieve ultrashort throw projection through a parameterized algorithm to stitch multiple images into one single frame, alleviating the trade‐off between the resolution and frame rate in conventional laser MEMS displays. To evaluate the effectiveness, we construct a test system that consists of three off‐the‐shelf projectors with a throw ratio of 1.3 and three planar reflective mirrors. Using our method, the integrated system yields an ultrashort throw ratio of 0.26, greatly reducing the projection distance from 12 to 2.6 ft for the 120‐in. screen in diagonal. Moreover, our algorithm can be fine‐tuned to correct for image distortion, intensity variation, and edge mismatch induced by two‐axis light beam steering, thereby further improving the image quality.     

Android平台上的多用户无线投影控制系统设计

运用WIFI无线通信技术、ARM Cortex A8处理器嵌入式系统、TCP/IP协议以及无线局域网监控等技术设计并制作了一个Android平台的多用户无线投影控制系统。通过WIFI无线通信技术建立用户端与无线投影网关的无线通信,解决了数据连线对投影仪使用时的限制,极大地方便了有线投影仪用户使用无线投影功能。本系统通过无线局域网实现多用户、多平台的无线投影,采用一主多从的方式进行切换,适用于PC机和Android手持移动设备。系统经测试,达到使用要求。本文还详细描述了无线投影网关设计和无线投影管理软件设计。     

Optical simulation for cross‐talk evaluation and improvement of autostereoscopic 3‐D displays with a projector array

Abstract— Multi‐view spatial‐multiplexed autostereoscopic 3‐D displays normally use a 2‐D image source and divide the pixels to generate perspective images. Due to the reduction in the resolution of each perspective image for a large view number, a super‐high‐resolution 2‐D image source is required to achieve 3‐D image quality close to the standard of natural vision. This paper proposes an approach by tiling multiple projection images with a low magnification ratio from a microdisplay to resolve the resolution issue. Placing a lenticular array in front of the tiled projection image can lead to an autostereoscopic display. Image distortion and cross‐talk issues resulting from the projection lens and pixel structure of the microdisplay have been addressed with proper selection of the active pixel and adequate pixel grouping and masking. Optical simulation has shown that a 37‐in. 12‐view autostereoscopic display with a full‐HD (1920 × 1080) resolution can be achieved with the proposed 3‐D architecture.     

A high‐resolution autostereoscopic display system with a wide viewing angle using an LCOS projector array

Abstract— This study develops an autostereoscopic display based on a multiple miniature projector array to provide a scalable solution for a high‐resolution 3‐D display with large viewing freedom. To minimize distortion and dispersion, and to maximize the modulation transfer function (MTF) of the projection image to optimize 3‐D image quality, a dedicated projection lens and an accurate six‐axis adjusting platform for the miniature projector were designed and fabricated. Image‐blending technology based on a lookup table was adopted to combine images from 30 miniature projectors into a seamless single image. The result was a 35‐in. autostereoscopic display with 60 views ata 30° viewing angle, 90° FOV, and large range of viewing distance. The proposed system exhibits very smooth motion parallax when viewers move around in front of it.     

微距摄影的多聚焦图像拍摄和融合

目的 对于微距摄影来说,由于微距镜头的景深有限,往往很难通过单幅照片获得拍摄对象全幅清晰的图像.因此要想获取全幅清晰的照片,就需要拍摄多幅具有不同焦点的微距照片,并对其进行融合.方法 传统的微距照片融合方法一般都假定需要融合的图像是已经配准好的,也并没有考虑微距图像的自动采集.因此提出了一种用于微距摄影的多聚焦图像采集和融合系统,该系统由3个部分组成.第1部分是一种微距图像拍摄装置,该硬件能够以高精度的方式拍摄物体在不同焦距下的微距照片.第2部分是一个基于不变特征的图像配准组件,它可以对在多个焦点下拍摄的微距图像进行自动配准和对齐.第3部分是一个基于图像金字塔的多聚焦图像融合组件,这个组件能够对已经对齐的微距照片进行融合,使得合成的图像具有更大的景深.该组件对基于图像金字塔的融合方法进行了扩展,提出了一种基于滤波的权重计算策略.通过将该权重计算与图像金字塔相结合,得到了一种基于多分辨率的多聚焦图像融合方法.结果 论文使用设计的拍摄装置采集了多组实验数据,用以验证系统硬件设计和软件设计的正确性,并使用主观和客观的方法对提出的系统进行评价.从主观评价来看,系统合成的微距图像不仅具有足够的景深,而且在高分辨率下也能够清晰地呈现物体微小的细节.从客观评价来看,通过将系统合成的微距图像与其他方法合成的微距图像进行量化比较,在标准差、信息熵和平均梯度3种评价标准中都是最优的.结论 实验结果表明,该系统是灵活和高效的,不仅能够对多幅具有不同焦点的微距图像进行自动采集、配准和融合,并且在图像融合的质量方面也能和其他方法相媲美.     

High‐pixel‐rate grating‐light‐valve laser projector

Abstract— A high‐pixel‐rate, high‐contrast (30,000:1) wide‐color‐gamut grating‐light‐valve laser projector is reported. A new optical engine enabling high‐frame‐rate (240 Hz) scan projection is employed. Panoramic wide‐angle‐scan projection with a 64:9 aspect ratio was also developed. Speckle noise is eliminated using a simple but highly efficient technique. The optical throughput efficiency of the grating‐light‐valve laser projector is reviewed.