多视点三维投影显示系统

许多人都认为自动分光立体显示技术将使得家庭视觉体验朝着更自然、更逼真的方向发展.为了营造更逼真、更刺激的 VR 效果,屏幕越大越好.本文旨在详细介绍运用在 3D 显示的多视点三维投影系统.三维图像方法的原理是视差的深度暗示.采用的方法的视差不是由于使用特殊光学眼镜或者是特殊的光学头盔作为观察光学图像形成区域而引起的.实验装置系统包括暗箱组、PC 组和一个多视点三维显示器.采用了透镜、视差栅栏来制作空间多元多视角的自动分光立体显示器.使用投影仪来模拟分光图像对.这种自动分光显示屏幕的优点在于:容易按比例放大;低成本;投影仪和屏幕间不需要精确对准.本文还研究了该屏幕的制作工艺,并给出了实验结果.     

Image-Forming Principle of Integral Photography

In this paper, viewing zone-forming geometry of multiview imaging systems and lens image forming principle is applied to prove that the depth sensing mechanism of integral photography (IP) is both parallaxes. The proof is based on the fact that making the cell size of each elemental image the same as a microlens pitch does not change the geometry, because there are no changes in the field-of-view of each microlens. The total number of different view images perceived in the viewing zone of IP, and the compositions of the images and condition of increasing the number are also identified.      

Methods for Displaying Three-Dimensional Images

The essential components of generating three-dimensional (3-D) images are defined, and various methods of creating each of the components are explained. The components are image acquisition, image multiplexing and processing, and display mechanisms. For image acquisition, transforming, synthesizing based on disparity, voxel and wavefront deformation, and sampling and photographing methods are used. For multiplexing, time, spatial, and spatiotemporal, and for processing, chirping, layering, and pixel cell based multiview image arrangement methods are used. The display mechanisms are classified into projection, contact,and scanning types depending on their means of displaying images. From these methods, 3-D images with real volume, with parallax only and with psychologically induced depth sense can be generated. The display mechanisms have another important mission of creating viewing zones. For this purpose, all 3-D imaging methods employ a special form of optics or mechanisms complying with their image multiplexing schemes. These optics and mechanisms are essential in realizing 3-D imaging systems but at the same time they provide many unfriendly and uncomfortable effects to viewers, and also impose some functional limitations.     

Resampling, Antialiasing, and Compression in Multiview 3-D Displays

Multiview three-dimensional (3-D) displays offer viewing of high-resolution stereoscopic images from arbitrary positions without glasses. This article surveyed different approaches to develop signal processing algorithms for these displays. Such displays consist of view-dependent pixels that reveal a different color according to the viewing angle. Therefore, the left and right eye of an observer sees slightly different images on the screen. This leads to the perception of 3-D depth and parallax effects when the observer moves. Although the basic optical principles of multiview auto-stereoscopy have been known for over a century, only recently displays with increased resolution, or systems based on multiple projectors, have made this approach practical.     

Preclinical ROC studies of digital stereomammography

Reports the diagnostic performance of observers in detecting abnormalities in computer-generated mammogram-like images. A mathematical model of the human breast is defined in which breast tissues are simulated by spheres of different sizes and densities. Images are generated by casting rays from a specified source, through the model, and onto an image plane. Observer performance when using two viewing modalities (stereo versus mono) is compared. In the stereo viewing mode, images are presented to the observer (wearing liquid-crystal display glasses), such that the left eye sees the left image only and the right eye sees the right image only. In this way, the images can be fused by the observer to obtain a sense of depth. In the mono viewing mode, identical images are presented to the left and right eyes so that no binocular disparities will be produced by the images. Observer response data are evaluated using receiver operating characteristic (ROC) analysis to characterize any difference in detectability of abnormalities (in either the density or the arrangement of simulated tissue densities) using the two viewing modes. The authors' experimental results indicate the clear superiority of stereo viewing for detection of arrangement abnormalities. For detection of density abnormalities, the performance of the two viewing modes is similar. These preliminary results suggest that stereomammography may permit easier detection of certain tissue abnormalities, perhaps providing a route to earlier tumor detection in cases of breast cancer.     

基于视觉显著性图的立体图像视疲劳评价

随着立体显示技术的飞速发展,立体图像带来的视疲劳问题引发了人们的广泛关注.视觉显著性区域决定着立体图像的性质,而双目视差和观看时间是影响立体图像视疲劳程度的两个重要因素.为了评价观看立体图像后产生的视疲劳,提出了立体图像视疲劳的评价模型.该模型基于视觉显著性图,根据视差和观看时间得到对立体图像视疲劳的评价.生理心理实验表明,该模型预测结果的可靠率到达了93％以上,依据该评价模型可由视差和观看时间快速、准确地评价立体图像视疲劳,为立体图像视疲劳的评价及立体图像的改进提供了依据.     

No-reference Stereoscopic Image Quality Assessment Using Binocular Self-similarity and Deep Neural Network

Quality assessment of three-dimensional (3D) images is more challenging than that of 2D images. The quality of 3D visual experience is one of the most challenging areas of human binocular perception and is affected by multiple factors such as asymmetric stereo image/video compression, depth perception, visual discomfort, and single view quality. In this paper, we propose a new no-reference quality assessment method for stereoscopic images based on Binocular Self-similarity (BS) and Deep Neural Networks (DNN). To be more specific, a BS index is defined and computed according to binocular rivalry and suppression based on the depth image-based rendering technique. Then, a DNN is trained in an opinion unaware way to predict local quality. Binocular integration (BI) index is calculated by using the trained DNN, accounting for binocular integration behaviors. Finally, the final quality score of stereoscopic image is obtained by combining the BS and BI indexes together. Experimental results on four public 3D image quality assessment databases demonstrate that compared with existing methods, the proposed method can achieve high consistency with subjective perception on stereoscopic images with both symmetric and asymmetric distortions.     

Disparity field and depth map coding for multiview 3D image generation

In the present paper techniques are examined for the coding of the depth map and disparity fields for stereo or multiview image communication applications. It is assumed that both the left and right channels of the multiview image sequence are coded using block- or object-based methods. A dynamic programming algorithm is used to estimate a disparity field between each stereo image pair. Depth is then estimated and occlusions are optionally detected, based on the estimated disparity fields. Spatial interpolation techniques are examined based on the disparity/depth information and the detection of occluded regions using either stereoscopic or trinocular camera configurations. It is seen that the presence of a third camera at the transmitter site improves the estimation of disparities, the detection of occlusions and the accuracy of the resulting spatial interpolation at the receiver. Various disparity field and depth map coding techniques are then proposed and evaluated, with emphasis given to the quality of the resulting intermediate images at the receiver site. Block-based and wireframe modeling techniques are examined for the coding of isolated depth or disparity map information. Further, 2D and 3D motion compensation techniques are evaluated for the coding of sequences of depth or disparity maps. The motion fields needed may be available as a byproduct of block-based or object-based coding of the intensity images. Experimental results are given for the evaluation of the performance of the proposed coding and spatial interpolation methods.     

A perceptual stereoscopic image quality assessment model accounting for binocular combination behavior

Stereoscopic image quality assessment (SIQA) plays an important role in the development of 3D image processing. In this paper, a full-reference object SIQA model is built based on binocular summation channel and binocular difference channel. In our frame work, binocular combination behavior and how to experience the depth perception are thought to be the key factors to evaluate the quality of stereoscopic images. Differing from the current depth map methods, this method focuses on a new aspect, and an effective combination model is proposed based on the physiological findings in the Human Visual System (HVS). Experimental results demonstrate that the proposed quality assessment metric significantly outperforms the existing metrics and can achieve higher consistency with subject quality assessment when predicting the quality of stereoscopic images that have been symmetrically distorted.     

一种3D视频中立体字幕叠加技术

运用3D字幕视差与背景景深的对应关系,基于立体视觉的双目视差原理叠加3D字幕,实现了基于三维图像内容的景深叠加3D字幕的系统.该立体字幕叠加方法具有简单、稳健等特点,缓解了观看时的视觉疲劳问题,使观众更舒适地观看立体图像和字幕.     

Repeated vergence adaptation causes the decline of visual functions in watching stereoscopic television

To evaluate visual fatigue when viewing stereoscopic TV, a technology expected to become the broadcasting display system of the future. Wide public acceptance of stereoscopic TV awaits resolution of many issues, including visual fatigue on viewing TV images. Visual fatigue was induced using a visual function simulator, consisting of prism and lens optical systems, while viewing stereoscopic TV. We assessed subject visual fatigue through subjective reports of symptoms and by the changes in visual functions. These functions included: viewer B [Js fusional break point, recovery point, accommodation step response, and visual evoked cortical potentials (VECP)]. Significant changes of some visual functions were found after watching simulated stereoscopic TV when the vergence load was heavy or when it changed over time; relative vergence limits decreased and the latency of VECP increased after watching, reflecting visual fatigue. After subjects rested, relative vergence limits recovered to pre-viewing levels. Our findings lead us to conclude that, aside from excessive horizontal binocular parallax, discontinuous changes in parallax is also a major factor that contributes to visual fatigue in the viewing of stereoscopic images. It also causes a decreased range of relative vergence, accommodation response, and a delay in the P100 latency of VECP.     

基于双目能量模型的立体图像质量客观评价方法

立体图像的深度感知取决于双目差距,基础视觉皮层的差距调谐细胞在感知立体深度的过程中起着关键作用.单只眼睛感知的图像信息被单目简单细胞接收之后传入双目简单细胞进行处理,继而被送入复杂细胞进行综合,得到基础视觉皮层对于一幅立体图像的能量响应.用数学模型来仿真简单的以及复杂的细胞对于双目差距的响应,并通过双目能量模型的计算达到评价彩色立体图像的目的.实验结果表明,双目能量模型的评价结果与立体图像的主观评价值具有较高的一致性.     

Cancellation of image crosstalk in time-sequential displays ofstereoscopic video

Stereoscopic visualization systems based on liquid crystal shutter (LCS) eyewear and cathode-ray tube (CRT) displays provide today the best overall quality of three-dimensional (3-D) images and therefore have a dominant position in commercial as well as professional markets. Due to the CRT and LCS characteristics, however, such systems suffer from perceptual crosstalk ("shadows") at object boundaries that can reduce, and at times inhibit, the ability to perceive depth. In this paper, we propose a method to reduce such crosstalk. We present a simple model for intensity leak, we assess model parameters for a time-sequential LCS/CRT system and we propose a computationally efficient algorithm to eliminate the crosstalk. Since the full crosstalk elimination implies an unacceptable image degradation (reduction of contrast), we study the tradeoff between crosstalk elimination and image contrast. We describe experiments on synthetic and natural stereoscopic images and we discuss informal subjective viewing of processed images. Overall, the viewer response has been very positive; 3-D perception of many objects became either much easier or even effortless. Since the proposed algorithm can be easily implemented in real time (only linear scaling and table look-up are needed), we believe that it can be successfully used today in various stereoscopic applications suffering from image crosstalk. This is particularly true for PC-based 3-D viewing where the algorithm can be executed by the CPU or by an advanced graphics board.     

柱透镜光栅3D显示器的视差范围与立体观看视疲劳的关系

过大的视差取值是产生柱透镜光栅3D显示器立体观看视疲劳的主要因素之一。基于人眼瞳孔直径随着立体观看视疲劳程度的增加而递增这一生理机理,本文建立了一个瞳孔直径的测量装置,可在观看者观看立体图像的同时记录观看者的瞳孔变化,从而客观评价观看者的立体观看视疲劳程度,并通过与观看者的主观评价相结合,得出不同视差大小下的立体观看视疲劳程度。实验结果表明,当视差在-0.2°～0.2°范围内时,观看者没有明显的立体观看视疲劳,可将该范围确定为观看舒适的视差范围。     

基于BJND的立体图像篡改定位及恢复水印方法

针对立体图像内容的真实性认证和完整性保护等 问题,提出一种基于双目恰可觉察失 真(BJND)的立体图像 篡改定位及恢复水印方法。首先,利用奇异值的稳定特性,设计左右图像的定位水印;然后 ,根据BJND强度将左右图像 的定位水印嵌入到左图像中;最后,利用离散余弦变换(DCT)和JPEG量化压缩 生成恢复信息,并将右图像和左图像遮挡暴露区 域的恢复信息分别嵌入到右图像和左图像中。实验结果表明,本文方法所构造的水印对JPEG 压缩、椒盐噪声、高斯白噪声等偶然攻击比较鲁棒,而对剪切、拼接、旋转等恶意攻击较为 脆弱,同时对左右图像的恶意篡改区域检测率大于98%。由于充分利用了左右图像的匹配特 性,恢复的被篡改立体图像其左右图像PSNR达36.02dB ～41.29dB。     

Hole‐Filling Methods Using Depth and Color Information for Generating Multiview Images

This paper presents new hole‐filling methods for generating multiview images by using depth image based rendering (DIBR). Holes appear in a depth image captured from 3D sensors and in the multiview images rendered by DIBR. The holes are often found around the background regions of the images because the background is prone to occlusions by the foreground objects. Background‐oriented priority and gradient‐oriented priority are also introduced to find the order of hole‐filling after the DIBR process. In addition, to obtain a sample to fill the hole region, we propose the fusing of depth and color information to obtain a weighted sum of two patches for the depth (or rendered depth) images and a new distance measure to find the best‐matched patch for the rendered color images. The conventional method produces jagged edges and a blurry phenomenon in the final results, whereas the proposed method can minimize them, which is quite important for high fidelity in stereo imaging. The experimental results show that, by reducing these errors, the proposed methods can significantly improve the hole‐filling quality in the multiview images generated.     

基于三维感知的立体虚拟视点图像质量评价方法

为了评价立体虚拟视点图像的质量,提出了一种基 于三维感知的客观评价方法。综合考虑了立体虚拟视点图像两大最主要失真类型:单视点绘 制失真和立体视点不匹配失真。针对单视点绘制失真,先提取 当前视点失真图与无失真图的差异性区域,再针对该差异性区域计算平均结构相似度(MSSIM ),最后将左 右视点平均池化作为单目纹理特征值;针对立体视点不匹配失真,先对左右视点失真图分别 进行视差映射, 再提取映射图与该视点失真图的差异区域作为双目不匹配区域,然后针对不匹配区域计算MS SIM 值,最 后将左右视点平均池化作为双目竞争特征值;最终将两个特征值幂次融合,作为最终的立体 虚拟视点图像 质量评价客观指标。实验结果表明本方法有效匹配主观打分的DMOS值,皮尔森线性相关系数 和斯皮尔曼 秩相关系数分别为0.911和0.900,正确反映了 立体虚拟视点图像质量。     

基于光场结构特性与多视点匹配的深度估计

针对现有光场图像深度估计技术无法均衡地对主要对象和背景进行深度估计的问题,提出了一种基于光场结构特性与多视点匹配的深度估计方法。该方法在光场结构特性引导的深度估计的基础上,为了实现光场图像深度变化区域的平滑过渡,同时又考虑光场图像具有多视点子孔径图像阵列的特点,采用多视点匹配优化光场图像深度估计。在马尔可夫随机域中,基于光场结构特性构建深度估计平滑项,同时联合多视点匹配构建深度估计数据项,并进行全局深度迭代优化,从而有效平衡对象深度边界和背景深度估计,提高光场图像深度估计的性能。实验结果表明,所提出的方法能够得到更加清晰的深度边界,同时可以修正背景中不准确的深度值,获得高质量的深度估计结果。     

一种低成本的立体显示头跟踪装置

数字图像技术的最新发展将促进立体显示的实现 ,它使得在数字领域对立体图像进行高效处理和编码成为可能。立体图像显示已经应用于教育、医疗和娱乐等领域。反过来这些图像应用领域又促进了立体显示技术的发展。不管采用何种方式处理立体显示数据 ,因显示系统的终端为观察者 ,它必须以某种方式将正确的图像展现到观察者与其对应的眼前。一种简单经济的方法就是时分转换法。对应于左眼和右眼的图像在一个刷新周期内交替出现。观察者戴着光阀眼镜 ,当左眼图像显示时 ,左眼光阀被打开使图像通过 ,此时右眼光阀处于关闭状态阻止图像通过。当右眼图像显示时 ,光阀操作过程正好相反。这样 ,在立体显示中左眼只见到左眼图像 ,右眼只见到右眼图像。当观察者的眼睛位置随着其头部运动而发生变化时 ,应回馈其变化以调整左眼和右眼的图像使观察不受影响。此文介绍一种应用超声波传感器构成的简单的、低成本的显示头跟踪器。置于眼镜上的超声波发射器发送信号 ,其显示屏上 3个超声波接收器 (呈三角形放置 )接收信号。根据从发射信号到接收信号的时间间隔可以计算出头部位置。介绍了硬件结构、实验数据分析及校正算法。最后讨论了虚拟现实应用未来研究方向 ,即改善眼睛舒适度的自由观看系统。     

强散射背景下的图像感知、融合与可视化技术

视觉信息是人类对周边环境进行感知的重要手段,光学成像和图像处理技术能大大扩展人类“视域”,使得人们获取图像的方式不局限于眼睛能见范围。散射效应导致光学成像装置的作用距离大幅下降,难以对远距离目标进行有效观测。人类对图像信息的感知,通常由对焦、校正和立体视觉形成,三个步骤互相耦合完成。其中,对焦和双目图像信息校正过程可以通过光学系统和数字图像处理的方法进行优化,提高强散射背景下的图像对比度,进而使得散射条件下的图像信息得以被感知和分析。然而,在目前技术条件下,机器立体视觉仍难以达到人类视觉水平,而人类视觉系统仍然是图像感知和分析的重要终端。可以预见,在低能见度条件下实现光学图像信息的精确获取与分析,仍需要实现人类视觉系统和机器的双重结合,发展包含人类视觉在内的立体视觉全局优化技术。主要介绍了在大气和水下浑浊条件下实现光学成像和实现图像融合的物理极限和关键影响因素,并展望人类的立体视觉在提高光学成像能力方面的作用。