Real-time system for high-image resolution disparity estimation.

We present the hardware implementation of a simple, fast technique for depth estimation based on phase measurement. This technique avoids the problem of phase warping and is much less susceptible to camera noise and distortion than standard block-matching stereo systems. The architecture exploits the parallel computing resources of FPGA devices to achieve a computation speed of 65 megapixels per second. For this purpose, we have designed a fine-grain pipeline structure that can be arranged with a customized frame-grabber module to process 52 frames per second at a resolution of 1280 x 960 pixels. We have measured the system's degradation due to bit quantization errors and compared its performance with other previous approaches. We have also used different Gabor-scale circuits, which can be selected by the user according to the application addressed and typical image structure in the target scenario.     

Discontinuity preserving disparity estimation with occlusion handling

In this paper, we propose a stereo matching algorithm based on distance transform to generate high-quality disparity maps with occlusion handling. In general, pixel intensities around object edges are smeared due to mixed values located between the object and its background. This leads to problems when identifying discontinuous disparities. In order to handle these problems, we present an edge control function according to distance transform values. Meanwhile, occluded regions occur, i.e., some portions are visible only in one image. An energy function is designed to detect such regions considering warping, cross check, and luminance difference constraints. Consequently, we replace the disparity in the occluded region with the one chosen from its neighboring disparities in the non-occluded region based on color and spatial correlations. In particular, the occlusion hole is filled according to region types. Experimental results show that the proposed method outperforms conventional stereo matching algorithms with occlusion handling.     

One-dimensional dense disparity estimation for three-dimensional reconstruction

We present a method for fully automatic three-dimensional (3D) reconstruction from a pair of weakly calibrated images in order to deal with the modeling of complex rigid scenes. A two-dimensional (2D) triangular mesh model of the scene is calculated using a two-step algorithm mixing sparse matching and dense motion estimation approaches. The 2D mesh is iteratively refined to fit any arbitrary 3D surface. At convergence, each triangular patch corresponds to the projection of a 3D plane. The proposed algorithm relies first on a dense disparity field. The dense field estimation modelized within a robust framework is constrained by the epipolar geometry. The resulting field is then segmented according to homographic models using iterative Delaunay triangulation. In association with a weak calibration and camera motion estimation algorithm, this 2D planar model is used to obtain a VRML-compatible 3D model of the scene.     

Simultaneous disparity estimation and object segmentation using stochastic models

Stochastic models are proposed for simultaneous disparity estimation and object segmentation in the stereo images. The disparity and segment fields are explicitly modelled as Markov random fields, and are estimated in the energy minimisation process called stochastic diffusion. Some experiments show good segmentation results of foreground objects.     

Adaptive fractional motion and disparity estimation skipping in MV-HEVC

MV-HEVC can efficiently compress multiview video data captured from different viewpoints. To achieve high coding efficiency, it consists of not only inter coding but also interview coding. The inter coding includes a motion estimation (ME) process that reduces temporal redundancies between consecutive frames, and the interview coding performs a disparity estimation (DE) that reduces interview redundancies between neighboring views. As a result, MV-HEVC needs high encoding complexity to perform both ME and DE. In order to reduce the complexity, this paper proposes an adaptive fractional ME and DE skipping method in a partitioned inter prediction unit (PU) mode, based on a result of a 2 N × 2 N inter PU coding. Experimental results show that the proposed method efficiently reduces the encoding complexity with negligible coding loss, compared to conventional methods.     

基于三角形网格的立体视差估计和遮挡点检测混合算法

本文提出一种新的用于立体图像编码的视差估计和遮挡点检测混合算法.其中的视差估计方法利用极线约束条件,在缩小搜索范围的同时提高了视差估计的准确性.遮挡点检测方法仅使用了匹配点唯一性约束和视差梯度限制这两个基本条件,降低了算法的复杂度.整个算法利用DT(Dalaunay triangulation)网格这一数学工具把散乱的点结合起来进行处理,使算法在实现方面更加简单化.本文算法首先对立体图像对中的左图像进行DT网格剖分,把各三角形的顶点作为"特征点"在右图像中寻找它们的匹配点.然后利用匹配点唯一性条件提取出其中一些顶点进行遮挡检测.实验结果表明,本文算法对"特征点"的视差估计比较准确,也能较为准确地检测出其中的遮挡点.借助DT 网格在图像编码方面的优势,本文算法可以方便地用于立体图像编码.     

基于差异的视差估计方法研究

视差估计在立体图像编码、视频检索、立体图像分割等领域有重要的应用,是进行图像、视频分析的基础。首先,基于模糊隶属度提出了一种基于差异的视差估计方法;其次,引入模糊隶属度,把图像灰度转换为差异值;然后,分析了多块匹配的优越性;提出了基于多块匹配的视差估计方法;最后,在Middlebury网站上的测试结果证明了该方法的有效性。     

基于人工视觉对比度掩模的鲁棒图像融合系统

现有的图像融合算法,不能有效区分噪声和视觉上有意义的特征,往往在输入含有噪声时性能变差。人类视觉系统对图像特征提取有特殊的机制。本文利用人类视觉系统对比度掩模过程对噪声和边缘信息的识别机制,在视觉域进行图像融合,达到自适应抑制噪声,增强图像特征的目的。对各方案性能的主、客观比较,充分验证了本文方案的优越性。     

Dynamically adaptive real-time disparity estimation hardware using iterative refinement

The computational complexity of disparity estimation algorithms and the need of large size and bandwidth for the external and internal memory make the real-time processing of disparity estimation challenging, especially for High Resolution (HR) images. This paper proposes a hardware-oriented adaptive window size disparity estimation (AWDE) algorithm and its real-time reconfigurable hardware implementation that targets HR video with high quality disparity results. Moreover, an enhanced version of the AWDE implementation that uses iterative refinement (AWDE-IR) is presented. The AWDE and AWDE-IR algorithms dynamically adapt the window size considering the local texture of the image to increase the disparity estimation quality. The proposed reconfigurable hardware architectures of the AWDE and AWDE-IR algorithms enable handling 60 frames per second on a Virtex-5 FPGA at a 1024×768 XGA video resolution for a 128 pixel disparity range.     

Edge-preserving disparity estimation and disparity-compensated intermediate view reconstruction for stereo images

A constrained disparity estimation method is proposed which uses a directional regularization technique to efficiently preserve edges for stereo image coding. The proposed method smoothes disparity vectors in smooth regions and preserves edges in object boundaries well, without creating an oversmoothing problem. The differential pulse code modulation (DPCM) technique for disparity map coding is used prior to entropy coding, in order to improve the overall coding efficiency. The proposed disparity estimation method can also be applied to intermediate view reconstruction. Intermediate views between a left image and a right image provide reality and natural motion parallax to multiviewers. Intermediate views are synthesized by appropriately exploiting an interpolation or an extrapolation technique according to the characteristics of each region after identifying the regions as occluded regions, normal regions, and regions having ambiguous disparities.The experimental results show that the proposed disparity estimation method gives close matches between a left image and a right image and improves coding efficiency. In addition, we can subjectively confirm that the application of our proposed intermediate view reconstruction method leads to satisfactory intermediate views from a stereo image pair.This work was supported by the Korea Institute of Science and Technology (KIST) under Grant No. 99HI-054.     

立体图像分层交叠块视差估计与自适应补偿

视差估计与补偿是立体图像编码的关键问题,本文提出了一种基于交叠块匹配的视差估计与补偿算法。基于交叠块匹配的分层视差估计,将块分割和平滑约束融入视差估计,可得到光滑准确的视差矢量场。自适应交叠块视差补偿可克服通常的交叠块视差补偿的过平滑效应,在相同的编码比特率下,与块补偿及OBDC相比,AOBDC可获得2.4 dB及0.9 dB的峰值信噪比增益。     

Human visual system based data embedding method using quadtree partitioning

In this paper, we proposed a data embedding method based on human visual system (HVS) and quadtree partitioning. For most HVS-based methods, the amount of embedded data is based on the measurement of differences of pixel pairs or the standard deviation of image blocks. However, these methods often result in larger image distortion and are vulnerable to statistical attacks. The proposed method employs a specially designed function to measure the complexity of image blocks, and uses quadtree partitioning to partition images into blocks with different sizes. Larger blocks are associated with smooth regions in images whereas smaller blocks are associated with complex regions. Therefore, we embed less data into larger blocks to preserve the image quality and embed more data into smaller blocks to increase the payload. Data embedment is done by using the diamond encoding technique. Experimental results revealed that the proposed method provides better image quality and offers higher payload compared to other HVS-based methods.     

Parallel framework for dense disparity map estimation using Hamming distance

A novel framework for sparse and dense disparity estimation was designed, and the proposed framework has been implemented in CPU and GPU for a parallel processing capability. The Census transform is applied in the first stage, and then, the Hamming distance is later used as similarity measure in the stereo matching stage followed by a matching consistency check. Next, a disparity refinement is performed on the sparse disparity map via weighted median filtering and color K-means segmentation, in addition to clustered median filtering to obtain the dense disparity map. The results are compared with state-of-the-art frameworks, demonstrating this process to be competitive and robust. The quality criteria used are structural similarity index measure and percentage of bad pixels (B) for objective results and subjective perception via human visual system demonstrating better performance in maintaining fine features in disparity maps. The comparisons include processing times and running environments, to place each process into context.     

Fast motion and disparity estimation for HEVC based 3D video coding

The emerging international standard for high efficiency video coding (HEVC) based 3D video coding (3D-HEVC) is an extension of HEVC. In the test model of 3D-HEVC, variable size motion estimation (ME) and disparity estimation (DE) are both employed to select the best coding mode for each treeblock in the encoding process. This technique achieves the highest possible coding efficiency, but it brings extremely high computational complexity which limits 3D-HEVC from practical applications. In this paper, a fast ME/DE algorithm based on inter-view and spatial correlations is proposed to reduce 3D-HEVC computational complexity. Since the multi-view videos represent the same scene with similar characteristic, there is a high correlation among the coding information from inter-view prediction. Besides, the homogeneous regions in texture video have a strong spatial correlation, and thus spatially neighboring treeblocks have similar coding information. Therefore, we can determine ME search range and skip some specific ME and DE rarely used in the previously coded view frames and spatially neighboring coding unit. Experimental results demonstrate that the proposed algorithm can significantly reduce computational complexity of 3D-HEVC encoding while maintaining almost the same rate-distortion performance.     

Fast technique for phase-based disparity estimation with noexplicit calculation of phase

A simpler and faster technique for depth estimation, based on phase measurements of disparity, is presented. The technique provides direct evaluations of phase differences, avoids explicit calculations of single phases and the attendant problem of phase wrapping, and is suitable for efficient software and hardware implementations     

Phase estimation of visual evoked responses

The phase of visual evoked responses (VERs) is one of the basic parameters in functional diagnostics of the visual system. A new method for phase estimation of VERs based on the observer model in system identification is introduced. Simulated data show significantly less variance of estimation than actual estimators do. By means of the new estimator, the dynamics of the visual system according to selected optical stimuli has been analyzed.     

Plane surface detection and reconstruction using segment-based tensor voting

A Segment-based Tensor Voting (SBTV) algorithm is presented for planar surface detection and reconstruction of man-made objects. Our work is inspired by piecewise planar stereo reconstruction. During the vital procedure to detect and label the planar surface, the two main contributions are: first, tensor voting is used for obtaining the geometry attribute of the 3D points cloud. The candidate planar patches are generated through scene image segment of low variation of color and intensity. Second, we over-segment the scene image into the segment and the candidate 3D planar patch is generated. The SBTV algorithm is used on 3D points cloud sets to identify the co-plane on the candidate patch. After detecting every planar patch, the geometry architecture of object is obtained. The experiments demonstrate the effectiveness of our proposed approach on either outdoor or indoor datasets.     

Limited yield estimation for visual defect sources

Although kill rate, kill ratio, and limited yield for visual defects are useful concepts in yield management, the formal definitions of these concepts, how to estimate them, especially in the presence of inspection error, and assumptions necessary for their application, are lacking in the literature. The concept of limited yield as the effect of a visual defect source on overall yield of a process is formally derived and the product of the individual limited yields for the visual defect sources is shown to equal the overall yield of the process. As a result of a more rigorous definition of limited yield, a major simplification in the calculation of limited yield over other methods is obtained. Basic to the notion of limited yield are the concepts of kill rate and kill ratio. The kill rate expresses how likely it is that a die with a certain visual defect will be rejected at probe. The kill ratio is shown to be the increased chance, relative to the baseline yield, of a die being rejected when a particular visual defect type is present. The limited yield concept is discussed and illustrated with a practical example using semiconductor visual defect data     

Modeling and estimation problems in the turtle visual cortex

The goal of this paper is to verify that position and velocity of a spot of light incident on the retina of a turtle are encoded by the spatiotemporal dynamics of the cortical waves they generate. This conjecture is examined using a biophysically realistic large-scale computational model of the visual cortex implemented with the software package, GENESIS. The cortical waves are recorded and analyzed using principal components analysis and the position and velocity information from visual space is mapped onto an abstract B-space, to be described, using the coefficients of the principal components expansion. The likely values of the position/velocity are estimated using standard statistical detection methods.