Stereoscopic view synthesis based on region-wise rendering and sparse representation

Depth image-based rendering (DIBR), which is used to render virtual views with a color image and the corresponding depth map, is one of the key techniques in the 2D to 3D conversion process. One of the main problems in DIBR is how to reduce holes that occur on the generated virtual view images. In this paper, we make two main contributions to deal with the problem. Firstly, a region-wise rendering framework, which divides the original image regions into three special classes and renders each with optimal adaptive process respectively, is introduced. Then, a novel sparse representation-based inpainting method, which can yield visually satisfactory results with less computational complexity for high quality 2D to 3D conversion, is proposed. Numerical experimental results demonstrate the good performance of the proposed methods.     

Full-reference quality assessment of stereopairs accounting for rivalry

We develop a framework for assessing the quality of stereoscopic images that have been afflicted by possibly asymmetric distortions. An intermediate image is generated which when viewed stereoscopically is designed to have a perceived quality close to that of the cyclopean image. We hypothesize that performing stereoscopic QA on the intermediate image yields higher correlations with human subjective judgments. The experimental results confirm the hypothesis and show that the proposed framework significantly outperforms conventional 2D QA metrics when predicting the quality of stereoscopically viewed images that may have been asymmetrically distorted.     

An efficient depth image-based rendering with depth reliability maps for view synthesis

In this paper, an efficient depth image-based rending (DIBR) with depth reliability maps (DRM) is proposed to improve the quality of synthesized images. First, a DRM-based occlusion-aware approach is developed to obtain a segmentation mask, which can explicitly indicate where the information in an intermediate image should be blended preferably. Next, an improved weight model for view creation is introduced to enhance the quality of synthesized images. Finally, a distance and depth-based sub-pixel weighted (DDSPW) algorithm is presented to solve the visibility and resampling problems. Experimental results demonstrate that the treated DIBR schemes have better performance for view synthesis than the other three methods through the subjective visual perception and objective assessments in terms of peak signal to noise ratio and structural similarity index.     

基于视觉感知和零水印的部分参考立体图像质量评价模型

通过分析人类视觉系统的纹理方向特性和立体感知特性,并结合数字水印的半脆弱性和支持向量回归(Support Vector Regression, SVR)的泛化学习能力,该文提出一种基于视觉感知和零水印的部分参考立体图像质量客观评价模型。该模型利用立体图像左右视点经小波分解后在同一空间频率的水平和垂直方向子带系数关系构造反映图像纹理方向特征的视点零水印,同时,利用左右视点视差值与自适应阈值的大小关系构造反映立体感质量的视差零水印,然后利用SVR来学习两类零水印恢复率(视觉加权视点零水印恢复率和视差零水印恢复率)与主观评价值的关系,最后用训练好的SVR完成立体图像质量预测。实验结果表明该模型符合人眼视觉特性,所得到的客观评价值与主观评价值具有较好的一致性。     

基于图像偏移角和多分支卷积神经网络的旋转不变模型设计

卷积神经网络(CNN)具有平移不变性,但缺乏旋转不变性。近几年,为卷积神经网络进行旋转编码已成为解决这一技术痛点的主流方法,但这需要大量的参数和计算资源。鉴于图像是计算机视觉的主要焦点,该文提出一种名为图像偏移角和多分支卷积神经网络(OAMC)的模型用于实现旋转不变。首先检测输入图像的偏移角,并根据偏移角反向旋转图像;将旋转后的图像输入无旋转编码的多分支结构卷积神经网络,优化响应模块,以输出最佳分支作为模型的最终预测。OAMC模型在旋转后的手写数字数据集上以最少的8 k参数量实现了96.98%的最佳分类精度。与在遥感数据集上的现有研究相比,模型仅用前人模型的1/3的参数量就可将精度最高提高8%。     

A perceptual stereoscopic image quality assessment model accounting for binocular combination behavior

Stereoscopic image quality assessment (SIQA) plays an important role in the development of 3D image processing. In this paper, a full-reference object SIQA model is built based on binocular summation channel and binocular difference channel. In our frame work, binocular combination behavior and how to experience the depth perception are thought to be the key factors to evaluate the quality of stereoscopic images. Differing from the current depth map methods, this method focuses on a new aspect, and an effective combination model is proposed based on the physiological findings in the Human Visual System (HVS). Experimental results demonstrate that the proposed quality assessment metric significantly outperforms the existing metrics and can achieve higher consistency with subject quality assessment when predicting the quality of stereoscopic images that have been symmetrically distorted.     

Discontinuity preserving disparity estimation with occlusion handling

In this paper, we propose a stereo matching algorithm based on distance transform to generate high-quality disparity maps with occlusion handling. In general, pixel intensities around object edges are smeared due to mixed values located between the object and its background. This leads to problems when identifying discontinuous disparities. In order to handle these problems, we present an edge control function according to distance transform values. Meanwhile, occluded regions occur, i.e., some portions are visible only in one image. An energy function is designed to detect such regions considering warping, cross check, and luminance difference constraints. Consequently, we replace the disparity in the occluded region with the one chosen from its neighboring disparities in the non-occluded region based on color and spatial correlations. In particular, the occlusion hole is filled according to region types. Experimental results show that the proposed method outperforms conventional stereo matching algorithms with occlusion handling.     

立体显示下表面光泽感知研究现状与展望

随着立体显示技术的成熟,基于立体显示的图像增强方法成为研究热点。人眼光泽感知是一种独特的视觉特征,它反映了物体表面的物理特性,对人类的认知活动至关重要。在立体显示下,双目光泽的呈现与感知研究是双目视觉研究的有力扩展和补充,可为立体显示设备再现更为丰富的图像外观提供应用基础。本文从双目光泽的现象学、理论学和影响线索3个方面展开分析,总结了立体显示下双目光泽的研究现状,指出了当前双目光泽感知研究存在的问题：对目标产生光泽感知的深层神经机理尚不清楚;对影响光泽感知的线索探究还停留在现象表面,其背后的多线索交互作用仍不明确;光泽感知测量评价的定量心理物理学实验数据有待扩充。未来,基于立体显示技术的光泽感知与视觉系统的联合探索或能成为极具潜力的研究热点。由于光泽感知是一种心理量,每个人对它的感知都存在特异性,我们认为需要精心设计更多的光泽感知实验来研究立体显示器中目标表面光泽属性再现,以扩展立体显示下的基础表示场景。     

雷达像智能识别对抗研究进展

基于深度神经网络的雷达像智能识别技术已经成为雷达信息处理领域的前沿和热点。然而,深度神经网络模型易受到对抗攻击的威胁。攻击者可以在隐蔽的条件下误导智能目标识别模型做出错误预测,严重影响其识别精度和鲁棒性。该文梳理了近年来雷达像智能识别对抗技术发展现状,总结分析了现有雷达一维/二维像识别对抗攻击方法和防御方法的特点,最后讨论了当前雷达像智能识别对抗研究领域值得关注的5个开放问题。     

基于深度信息的立体视频错误隐藏方法

针对双视加深度结构的立体视频的网络传输,该文提出了解码端的错误检测和错误隐藏技术.结合深度信息对发生错误宏块的编码模式进行估计,在错误宏块周围相邻宏块的编码模式不一致时,选择邻近宏块中深度与待重建的错误宏块深度最为接近的宏块的编码模式作为其编码模式.然后,根据编码模式选择用视点内或视点间的相关性信息重建错误宏块.实验结果表明采用该方法重建的图像客观质量峰值信噪比(PSNR)较传统的边界匹配错误隐藏方法提高了0.8-6.0 dB,并且主观质量也有提升.     

Adaptive depth truncation filter for MVC based compressed depth image

In multiview video plus depth (MVD) format, virtual views are generated from decoded texture videos with corresponding decoded depth images through depth image based rendering (DIBR). 3DV-ATM is a reference model for the H.264/AVC based multiview video coding (MVC) and aims at achieving high coding efficiency for 3D video in MVD format. Depth images are first downsampled then coded by 3DV-ATM. However, sharp object boundary characteristic of depth images does not well match with the transform coding based nature of H.264/AVC in 3DV-ATM. Depth boundaries are often blurred with ringing artifacts in the decoded depth images that result in noticeable artifacts in synthesized virtual views. This paper presents a low complexity adaptive depth truncation filter to recover the sharp object boundaries of the depth images using adaptive block repositioning and expansion for increasing the depth values refinement accuracy. This new approach is very efficient and can avoid false depth boundary refinement when block boundaries lie around the depth edge regions and ensure sufficient information within the processing block for depth layers classification. Experimental results demonstrate that the sharp depth edges can be recovered using the proposed filter and boundary artifacts in the synthesized views can be removed. The proposed method can provide improvement up to 3.25 dB in the depth map enhancement and bitrate reduction of 3.06% in the synthesized views.