Dual regularization-based image resolution enhancement for asymmetric stereoscopic images

Asymmetric stereoscopic imaging technique utilizes a pair of lower-resolution and full-resolution images to reduce the data storage requirement of stereoscopic images and videos, while maintaining fairly good quality in 3D perception. This paper proposes a resolution enhancement approach to reconstruct the original full-resolution image for this asymmetric stereoscopic system setup. The proposed approach exploits a dual regularization scheme: (i) a saliency-based regularization function is proposed to adaptively adjust the degree of regularization based on the local content of the image; and (ii) an occlusion-sensitive regularization function is proposed to exploit the correlation between the observed lower-resolution image and the observed full-resolution image in the neighboring view. Experiments are conducted to justify that the proposed approach outperforms a few conventional approaches.     

A depth perception and visual comfort guided computational model for stereoscopic 3D visual saliency

With the emerging development of three-dimensional (3D) related technologies, 3D visual saliency modeling is becoming particularly important and challenging. This paper presents a new depth perception and visual comfort guided saliency computational model for stereoscopic 3D images. The prominent advantage of the proposed model is that we incorporate the influence of depth perception and visual comfort on 3D visual saliency computation. The proposed saliency model is composed of three components: 2D image saliency, depth saliency and visual comfort based saliency. In the model, color saliency, texture saliency and spatial compactness are computed respectively and fused to derive 2D image saliency. Global disparity contrast is considered to compute depth saliency. Particularly, we train a visual comfort prediction function to distinguish stereoscopic image pair as high comfortable stereo viewing (HCSV) or low comfortable stereo viewing (LCSV), and devise different computational rules to generate a visual comfort based saliency map. The final 3D saliency map is obtained by using a linear combination and enhanced by a “saliency-center bias” model. Experimental results show that the proposed 3D saliency model outperforms the state-of-the-art models on predicting human eye fixations and visual comfort assessment.     

No-reference Stereoscopic Image Quality Assessment Using Binocular Self-similarity and Deep Neural Network

Quality assessment of three-dimensional (3D) images is more challenging than that of 2D images. The quality of 3D visual experience is one of the most challenging areas of human binocular perception and is affected by multiple factors such as asymmetric stereo image/video compression, depth perception, visual discomfort, and single view quality. In this paper, we propose a new no-reference quality assessment method for stereoscopic images based on Binocular Self-similarity (BS) and Deep Neural Networks (DNN). To be more specific, a BS index is defined and computed according to binocular rivalry and suppression based on the depth image-based rendering technique. Then, a DNN is trained in an opinion unaware way to predict local quality. Binocular integration (BI) index is calculated by using the trained DNN, accounting for binocular integration behaviors. Finally, the final quality score of stereoscopic image is obtained by combining the BS and BI indexes together. Experimental results on four public 3D image quality assessment databases demonstrate that compared with existing methods, the proposed method can achieve high consistency with subjective perception on stereoscopic images with both symmetric and asymmetric distortions.     

Panum融合区条件下的立体观影区间

由于生理的局限,人眼立体感知存在一定的盲区,并对立体视差存在一定的融合极限,因此获取符合人眼立体感知的三维影像就显得尤为重要.针对该问题,结合Panum融合区理论以及人眼立体感知误差,对立体观影区间的范围进行界定,并对立体观影区对应下的基线长度范围进行推导.根据推导结果,利用3ds max进行场景模拟,分别以特定距离下基线长度所允许的最大值和最小值为基准,获取5组不同基线长度下的视差图,最后利用虚拟现实眼镜验证了推导结果的正确性,该结果可对立体摄影过程的相机摆放提供指导.     

基于单片DMD的裸眼立体显示的实现方法研究

目前基于平板显示的立体显示设备有很多,比如等离子显示(PDP)和液晶显示(LCD).这些显示设备在超过 50 英寸时,价格都非常昂贵.本文提出了一种新的基于单片 DMD 的裸眼立体显示的实现方法.在分析DLP的显示原理和利用水平视差产生立体图像的机理以及系统构件(高压汞灯、色轮、数字微镜、透镜组、屏幕等)的传输特性的基础上,使用一种算法,在 FPGA 中把输入的视频图像分解为带有视差的图像序列,分别送到 DMD;同时产生视差同步信号,驱动执行部件.这样,在 DLP 显示屏前一定的区域内可以裸眼观察到立体图像.其特点是:单片 DMD 大屏幕显示;裸眼显示(不需佩戴特殊眼镜),立体景深可调;不损失器件的物理分辨率.     

双视立体视频同步合成系统设计与实现

以立体视频左右放置合成算法为核心算法,使用DDRSDRAM作为帧同步缓冲器件,采用FPGA作为平台,实现了一套立体视频合成系统。该系统降低了因制作立体视频需要处理两路带视差的视频而带来的对额外传输带宽和编解码器的需求,合成视频具有实时性好、失真小、立体感良好等特点。     

Depth-enhanced mobile robot teleguide based on laser images

3D stereoscopic visualization may provide a user with higher comprehension of remote environment in teleoperation when compared to 2D viewing. Works in the literature have addressed the contribution of stereo vision to improve perception of some depth cues often for abstract tasks, and it is hard to find contributions specifically addressing mobile robot teleguide. The authors of this paper have investigated stereoscopic viewing in mobile robot teleguide based on video images in a previous work and pointed out advantages of stereo viewing in this type of application as well as shortcomings inherent to the use of visual sensor, e.g. image transmission delay. The proposed investigation aims at testing mobile robot teleguide based on a different sensor: the laser sensor. The use of laser is expected to solve some problems related to visual sensor while maintaining the advantage of having stereoscopic visualization of a remote environment. A usability evaluation is proposed to assess system performance. The evaluation runs under the same setup of the previous study so to have an experimental outcome comparable to the previous one. The evaluation involves several users and two different 3D visualization technologies. The results show a strong improvement in users’ performance when mobile robot teleguide based on laser sensor is (depth-) enhanced by stereo viewing. Some differences are detected between the use of laser and visual sensor which are discussed.     

Mixed Display Platform to Expand Comfortable Zone of Stereoscopic 3D Viewing

Common stereoscopic three‐dimensional (3D) display has a convergence and accommodation conflict that violates the natural human cognitive process of viewing. This weakness exposes the challenge in supporting fun factors while eliminating safety problems in the 3D viewing experience. Thus, human factors have become a major research topic. In this letter, we propose a 3D stereoscopic visualization platform that can expand the sense of a 3D space by fusing organically mixed stereoscopic displays to provide a continuous feeling of 3D depth. In addition, we present pilot test results to show the possibility of the technical implementation of the proposed platform and note ongoing research issues to be addressed.     

A perceptual metric for stereoscopic image quality assessment based on the binocular energy

Stereoscopic imaging is becoming very popular and its deployment by means of photography, television, cinema. . .is rapidly increasing. Obviously, the access to this type of images imposes the use of compression and transmission that may generate artifacts of different natures. Consequently, it is important to have appropriate tools to measure the quality of stereoscopic content. Several studies tried to extend well-known metrics, such as the PSNR or SSIM, to 3D. However, the results are not as good as for 2D images and it becomes important to have metrics dealing with 3D perception. In this work, we propose a full reference metric for quality assessment of stereoscopic images based on the binocular fusion process characterizing the 3D human perception. The main idea consists of the development of a model allowing to reproduce the binocular signal generated by simple and complex cells, and to estimate the associated binocular energy. The difference of binocular energy has shown a high correlation with the human judgement for different impairments and is used to build the Binocular Energy Quality Metric (BEQM). Extensive experiments demonstrated the performance of the BEQM with regards to literature.     

3D多视点立体显示及其关键技术

作为基于 DTV/HDTV 的二维(2D)显示之后的下一代视频显示技术,三维(3D)多视点立体显示已成为国际上的研究热点之一.为建立多视点立体显示系统,阐述了相关的关键技术,包括:光场表示模型和光场获取系统、高效的与现行视频标准兼容的多视点编码和传输方法、解码端任意位置视点的高效绘制方法、3D显示技术以及多视点自由立体显示.针对上述关键技术,分析了当前国际上的发展趋势及存在的问题,同时提出了一种基于交互式自由立体显示的 3D 视频处理系统的解决方案.     

基于双眼视觉感知特征一致性的立体图像体验质量评价

3D图像被认为是多媒体技术的重要标志,其中,立体图像质量对3D图像发展起到至关重要的作用。不同于传统的2D图像质量评价,在3D图像质量评价中引入关于体验质量（ QoE）问题的新挑战,因此,本文提出一个基于双眼视觉感知特征一致性的立体图像体验质量评价算法。具体地,先对2个视点图像提取像素梯度作为视觉感知的低层次特征,再用梯度方向直方图特征（ HOG）建立立体图像的视觉感知特征向量,然后,由支持向量回归（ SVR）方法来学习视觉感知特征与立体图像体验质量得分的关系,最后,通过训练好的SVR模型来预测立体图像体验质量。实验结果表明所提算法能够有效地预测立体图像体验质量。     

偏振式微型三维投影系统串扰测量及分析

针对偏振式微型三维(3D)投影系统,研究了其主要性能参数———串扰对立体再现质量的影响。搭建了串扰测量平台,分别对偏振式立体投影仪、金属屏幕以及偏振眼镜3部分带来的串扰进行了测量及分析,得出3部分的串扰大小依次递减。分析结果表明,偏振式立体投影仪在减小串扰方面有待进一步提高,并印证了在立体显示中金属屏幕性能对系统再现质量起着决定性作用。本文为偏振式微型3D投影系统再现质量好坏的评价提供了性能参数及量化指标,也为偏振式立体投影仪性能参数研究及测量提供了方法。     

不同显示原理的眼镜式立体显示系统颜色特性分析

颜色特性是立体显示系统显示质量评估的重要指标之一。对于眼镜式立体显示系统来说,眼镜性能对显示效果的影响很关键。基于光电特性的测量,研究了不同原理的眼镜式立体显示系统的颜色特性。研究结果表明眼镜式立体显示系统在三维模式下,可显示颜色范围比二维模式下小60%以上。基于液晶开关眼镜的立体显示器在颜色特性方面表现效果不如基于偏振式眼镜的立体显示系统。     

Resampling, Antialiasing, and Compression in Multiview 3-D Displays

Multiview three-dimensional (3-D) displays offer viewing of high-resolution stereoscopic images from arbitrary positions without glasses. This article surveyed different approaches to develop signal processing algorithms for these displays. Such displays consist of view-dependent pixels that reveal a different color according to the viewing angle. Therefore, the left and right eye of an observer sees slightly different images on the screen. This leads to the perception of 3-D depth and parallax effects when the observer moves. Although the basic optical principles of multiview auto-stereoscopy have been known for over a century, only recently displays with increased resolution, or systems based on multiple projectors, have made this approach practical.     

Blind S3D image quality prediction using classical and non-classical receptive field models

We develop a novel no-reference image quality assessment model for stereoscopic 3D (S3D) images that is inspired by functional receptive field models of perceptual mechanisms in primary visual cortex (V1). The approach is called the Blind S3D Integrated Quality Evaluator (BSIQE). BSIQE simulates monocular and binocular responses to stereo views using channel separation and weighted multi-channel combination models. Binocular responses are modeled as the fusion of the two channels using a weighted multi-channel combination. The responses to stereoscopic image content of both classical and non-classical anisotropic receptive fields are then modeled based on a determination of the relative importance of the receptive field responses. In the last stage of feature extraction, we deploy a simple and efficient way of decorrelating the picture data. We extract local binary pattern (LBP) statistical features from the computed receptive field responses, and use them to train a regressor to predict the perceptual quality of stereoscopic images. We carefully evaluate BSIQE on four public-domain 3D image quality databases, and find that it is statistically superior to all compared 2D and 3D IQA algorithms. BSIQE exhibits good performance across the datasets suggesting that it is general, and it has relatively low complexity.     

助视3D显示技术概述

助视3D显示是指需要借助于3D眼镜等助视设备才能观看到3D图像的显示方式,它包括分色3D显示、偏振光3D显示和液晶快门3D显示。助视3D显示是目前最成熟的3D显示方式,近几年其相关技术及器件的性能得到了改进和提高。本文将介绍这三种助视3D显示技术。     

Reduced-reference stereoscopic image quality assessment based on view and disparity zero-watermarks

As a practical and novel application of watermarking, this paper presents a zero-watermarking based objective reduced-reference stereoscopic image quality assessment (RR-SIQA) method. In the proposed method, two kinds of zero-watermarks are constructed according to the characteristics of image structure and stereoscopic perception. Concretely, two view zero-watermarks, which are constructed by judging the relation of the horizontal and vertical components of gradient vectors with respect to the two views, are used to reflect the image structure variation of the stereoscopic image. Meanwhile, a disparity zero-watermark, which is constructed with disparity map of the stereoscopic image, is used to reflect the stereoscopic perception quality variation. Then, the quality of stereoscopic image is objectively assessed by pooling the recovering rates of the detected zero-watermarks. The experimental results show that the stereoscopic image quality evaluation results assessed with the proposed RR-SIQA method are well consistent with subjective assessment, and the proposed method achieves better performance than the widely used full-reference stereoscopic image quality assessment method PSNR in assessing quality of stereoscopic images compressed with JPEG and JPEG2000.     

No-reference stereoscopic image quality evaluator based on human visual characteristics and relative gradient orientation

Stereoscopic image quality assessment (SIQA) is of great significance to the development of modern three-dimensional (3D) display technology. In this work, by further mining the relationship between visual features and stereoscopic image quality perception, we build a new no-reference SIQA model, which combines the monocular and binocular features. Statistical quality-aware structural features from relative gradient orientation (RGO) map and texture features from the histogram of the weighted local binary pattern (LBP) in the texture image (TLBP) are not only extracted from both monocular view, but also extracted from binocular views to predict binocular quality perception. Meanwhile, the color statistical features ignored by most models and the binocularity feature is extracted to complement the monocular features and the above binocular features, respectively. Finally, all the extracted features and subjective scores are used to predict the objective quality score through the support vector regression (SVR) model. Experiments on four popular stereoscopic image databases show that the proposed model achieves high consistency with subjective assessment, and the performance of the model is very competitive with the latest models.     

A new objective stereoscopic image assessment model based on stereoscopic perception

In order to establish a stereoscopic image quality assessment method which is consistent with human visual perception, we propose an objective stereoscopic image quality assessment method. It takes into account the strong correlation and high degree of structural between pixels of image. This method contains two models. One is the quality synthetic assessment of left-right view images, which is based on human visual characteristics, we use the Singular Value Decomposition （SVD） that can represent the degree of the distortion, and combine the qualities of left and right images by the characteristics of binocular superposition. The other model is stereoscopic perception quality as- sessment, due to strong stability of image＇s singular value characteristics, we calculate the distance of the singular values and structural characteristic similarity of the absolute difference maps, and utilize the statistical value of the global error to evaluate stereoscopic perception. Finally, we combine two models to describe the stereoscopic image quality. Experimental results show that the correlation coefficients of the proposed assessment method and the human subjective perception are above 0.93, and the mean square errors are all less than 6.2, under JPEG, JP2K compression, Gaussian blurring, Gaussian white noise, H.264 coding distortion, and hybrid cross distortion. It indicates that the proposed stereoscopic objective method is consistent with human visual properties and also of availability.     

极端学习四元数小波特征的立体图像质量评价

为了有效、实时地对各种类型失真立体图像质量 进行评价,提出了一种基于极端学习(ELM) 和四元数小波交换(QWT)的无参考(NR)立体图像质量评价方法。首先利用SSI M密度立体匹 配模型生成相关的视差图、差异度可信图和右视图差异补偿图3D映射图;然后分别对左右视 图、视差图和差异度可信图进行 QWT,计算图像QWT第3相位系数相位幅值加权标准差和能量;再计算右视图 差异补偿图统 计特征熵和中值;最后将所提取的所有特征输入到基于核映射ELM 学习,预测失真立体图像 质量。在LIVE 3D图像质量评价数据库上的实验结果表明,本方法与人类主观质量评分具有较好的一致性 。在LIVE 3D图 像质量库I(Phase I)和库II(Phase II)上的斯皮尔曼相关系数(SROCC) 分别达到0.926和0.914