面向立体视频的视差-运动同步联立预测算法

针对立体视频序列的时间空间冗余和交叉冗余,提出一种快速编码算法.应用立体-运动约束模型在视差域和运动域进行联合迭代搜索,根据每次迭代后的模型误差设置自适应修正窗口来保证预测矢量精度,视差矢量和运动矢量可以快速同步获得.实验结果表明,该算法在保证高率失真性能的同时,可以大幅度降低编码复杂度.     

A Feature-based Approach for Dense Segmentation and Estimation of Large Disparity Motion

We present a novel framework for motion segmentation that combines the concepts of layer-based methods and feature-based motion estimation. We estimate the initial correspondences by comparing vectors of filter outputs at interest points, from which we compute candidate scene relations via random sampling of minimal subsets of correspondences. We achieve a dense, piecewise smooth assignment of pixels to motion layers using a fast approximate graphcut algorithm based on a Markov random field formulation. We demonstrate our approach on image pairs containing large inter-frame motion and partial occlusion. The approach is efficient and it successfully segments scenes with inter-frame disparities previously beyond the scope of layer-based motion segmentation methods. We also present an extension that accounts for the case of non-planar motion, in which we use our planar motion segmentation results as an initialization for a regularized Thin Plate Spline fit. In addition, we present applications of our method to automatic object removal and to structure from motion.     

Using Multiple-Hypothesis Disparity Maps and Image Velocity for 3-D Motion Estimation

In this paper we explore a multiple hypothesis approach to estimating rigid motion from a moving stereo rig. More precisely, we introduce the use of Gaussian mixtures to model correspondence uncertainties for disparity and image velocity estimation. We show some properties of the disparity space and show how rigid transformations can be represented. An algorithm derived from standard random sampling-based robust estimators, that efficiently estimates rigid transformations from multi-hypothesis disparity maps and velocity fields is given.     

基于运动和散焦线索的深度提取研究

随着立体图像在社会各个领域的发展与普及,三维资源的需求逐步提升,而将二维图像转换为三维图像是解决三维资源稀缺的重要手段。二维图像三维化技术的关键在于深度信息的获取。分析了两种重要的深度提取线索,运动线索和散焦模糊线索。为了弥补这两个线索在深度提取的局限性,提出了两种线索的深度融合策略。实验结果表明,这两种线索采用分段线性融合的方法能提高生成深度图的质量。     

Attentional Scene Segmentation: Integrating Depth and Motion

We present an approach to attention in active computer vision. The notion of attention plays an important role in biological vision. In recent years, and especially with the emerging interest in active vision, computer vision researchers have been increasingly concerned with attentional mechanisms as well. The basic principles behind these efforts are greatly influenced by psychophysical research. That is the case also in the work presented here, which adapts to the model of Treisman (1985, Comput. Vision Graphics Image Process. Image Understanding31, 156–177), with an early parallel stage with preattentive cues followed by a later serial stage where the cues are integrated. The contributions in our approach are (i) the incorporation of depth information from stereopsis, (ii) the simple implementation of low level modules such as disparity and flow by local phase, and (iii) the cue integration along pursuit and saccade mode that allows us a proper target selection based on nearness and motion. We demonstrate the technique by experiments in which a moving observer selectively masks out different moving objects in real scenes.     

一个优化运动矢量码率分配的快速算法

从信息论的角度看,运动补偿的最终目的是极小化信源熵率。因此,对具有四叉树结构的运动矢量场,当使用叶部运动矢量替代相应根部运动矢量作运动补偿预测时,若编码运动矢量所增加的熵大于相应补偿帧差所减少的熵,则删除相应的叶部运动矢量,反之,则应使用叶部运动矢量替代根部的运动矢量。依据这一准则,该文通过对正向搜索的运动矢量场进行自下而上反向优化截断,得到了一个全局优化运动矢量场码率分配的快速算法。通过假设补偿帧差系数的选验概率分布,该文首先给出了一个闭式补偿帧差系数熵理论计算公式,考虑到视频压缩的有损特性,然后又对这个闭式理论公式作了带权修正,为了进一步提高运动矢量的编码效率,文中同时也给出了一种运动矢量融合方案。最后,该文在运动补偿的三维小波视频编码系统上对这个算法进行了验证和比较。模拟实验证实该文所提出的运动矢量码率分配算法不仅速度快而且收敛性态稳定,只要提供合适的搜索深度,总可以获得最佳的分配结果。由于优化算法避开了补偿帧差的具体后处理,因此该文的算法也可以应用到传统的DCT视频编码器。     

基于运动估计的深度信息生成技术研究

国内外的多家电视媒体和拥有立体显示技术的企业都进行过开播数字立体频道的尝试,立体电视(3DTV)是未来的一个趋势.而立体视频片源制作费用高,周期长,是目前亟需解决的问题.描述了一种3DTV解决方案,提出了一个平面视频转换立体视频的方法.并主要针对方法中的深度信息生成问题,结合运动估计算法,解决了小范围内非关键帧深度信息自动生成问题,简化了转化过程中的人为工作量,提高了效率,降低了制作成本.证明了单目平面视频转换为基于深度信息的立体视频的可行性,得到较为满意的结果.     

Articulated Object Reconstruction and Markerless Motion Capture from Depth Video

We present an algorithm for acquiring the 3D surface geometry and motion of a dynamic piecewise‐rigid object using a single depth video camera. The algorithm identifies and tracks the rigid components in each frame, while accumulating the geometric information acquired over time, possibly from different viewpoints. The algorithm also reconstructs the dynamic skeleton of the object, thus can be used for markerless motion capture. The acquired model can then be animated to novel poses. We show the results of the algorithm applied to synthetic and real depth video.     

Focal Flow: Velocity and Depth from Differential Defocus Through Motion

We present the focal flow sensor. It is an unactuated, monocular camera that simultaneously exploits defocus and differential motion to measure a depth map and a 3D scene velocity field. It does this using an optical-flow-like, per-pixel linear constraint that relates image derivatives to depth and velocity. We derive this constraint, prove its invariance to scene texture, and prove that it is exactly satisfied only when the sensor’s blur kernels are Gaussian. We analyze the inherent sensitivity of the focal flow cue, and we build and test a prototype. Experiments produce useful depth and velocity information for a broader set of aperture configurations, including a simple lens with a pillbox aperture.     

Self-Supervised Monocular Depth and Motion Learning in Dynamic Scenes: Semantic Prior to Rescue

International Journal of Computer Vision - We introduce an end-to-end joint training framework that explicitly models 6-DoF motion of multiple dynamic objects, ego-motion, and depth in a monocular...     

Multi-level Motion Attention for Human Motion Prediction

International Journal of Computer Vision - Human motion prediction aims to forecast future human poses given a historical motion. Whether based on recurrent or feed-forward neural networks,...     

一种运动适应性快速运动估计算法

文中给出了一种适应性运动跟踪块匹配算法,利用相邻块的相关性确定搜索的初始点,根据相领块的运动性质确定当前块采用的搜索算法,实验表明此算法计算复杂度低,预测精度高。     

Depth Mapping for Stereoscopic Videos

Stereoscopic videos have become very popular in recent years. Most of these videos are developed primarily for viewing on large screens located at some distance away from the viewer. If we watch these videos on a small screen located near to us, the depth range of the videos will be seriously reduced, which can significantly degrade the 3D effects of these videos. To address this problem, we propose a linear depth mapping method to adjust the depth range of a stereoscopic video according to the viewing configuration, including pixel density and distance to the screen. Our method tries to minimize the distortion of stereoscopic image contents after depth mapping, by preserving the relationship of neighboring features and preventing line and plane bending. It also considers the depth and motion coherences. While depth coherence ensures smooth changes of the depth field across frames, motion coherence ensures smooth content changes across frames. Our experimental results show that the proposed method can improve the stereoscopic effects while maintaining the quality of the output videos.     

S-P表分析中差异系数计算的算法优化

S—P表分析法是计算机辅助教学中一种重要的信息处理方法。差异系数是S—P表分析中反映S曲线与P曲线之间不一致程度的重要指标。通过分析差异系数,可以获得大量关于被测对象整体情况的富有价值的信息,为教学情况的分析和教学效果的改善提供依据。鉴于目前已有的一些教育信息处理专著或文献资料中有关差异系数计算程序实现上的不足,本文提出并分析了两个优化算法,并给出了它们的流程图和源代码。     

Attention-Aware Disparity Control in interactive environments

Our paper introduces a novel approach for controlling stereo camera parameters in interactive 3D environments in a way that specifically addresses the interplay of binocular depth perception and saliency of scene contents. Our proposed Dynamic Attention-Aware Disparity Control (DADC) method produces depth-rich stereo rendering that improves viewer comfort through joint optimization of stereo parameters. While constructing the optimization model, we consider the importance of scene elements, as well as their distance to the camera and the locus of attention on the display. Our method also optimizes the depth effect of a given scene by considering the individual user’s stereoscopic disparity range and comfortable viewing experience by controlling accommodation/convergence conflict. We validate our method in a formal user study that also reveals the advantages, such as superior quality and practical relevance, of considering our method.     

Efficient Depth Propagation for Constructing a Layered Depth Image from a Single Image

In this paper, we propose an interactive technique for constructing a 3D scene via sparse user inputs. We represent a 3D scene in the form of a Layered Depth Image (LDI) which is composed of a foreground layer and a background layer, and each layer has a corresponding texture and depth map. Given user‐specified sparse depth inputs, depth maps are computed based on superpixels using interpolation with geodesic‐distance weighting and an optimization framework. This computation is done immediately, which allows the user to edit the LDI interactively. Additionally, our technique automatically estimates depth and texture in occluded regions using the depth discontinuity. In our interface, the user paints strokes on the 3D model directly. The drawn strokes serve as 3D handles with which the user can pull out or push the 3D surface easily and intuitively with real‐time feedback. We show our technique enables efficient modeling of LDI that produce sufficient 3D effects.     

Disparity analysis of images

An algorithm for matching images of real world scenes is presented. The matching is a specification of the geometrical disparity between the images and may be used to partially reconstruct the three-dimensional structure of the scene. Sets of candidate matching points are selected independently in each image. These points are the locations of small, distinct features which are likely to be detectable in both images. An initial network of possible matches between the two sets of candidates is constructed. Each possible match specifies a possible disparity of a candidate point in a selected reference image. An initial estimate of the probability of each possible disparity is made, based on the similarity of subimages surrounding the points. These estimates are iteratively improved by a relaxation labeling technique making use of the local continuity property of disparity that is a consequence of the continuity of real world surfaces. The algorithm is effective for binocular parallax, motion parallax, and object motion. It quickly converges to good estimates of disparity, which reflect the spatial organization of the scene.     

Depth estimation for ranking query optimization

A relational ranking query uses a scoring function to limit the results of a conventional query to a small number of the most relevant answers. The increasing popularity of this query paradigm has led to the introduction of specialized rank join operators that integrate the selection of top tuples with join processing. These operators access just “enough” of the input in order to generate just “enough” output and can offer significant speed-ups for query evaluation. The number of input tuples that an operator accesses is called the input depth of the operator, and this is the driving cost factor in rank join processing. This introduces the important problem of depth estimation, which is crucial for the costing of rank join operators during query compilation and thus for their integration in optimized physical plans. We introduce an estimation methodology, termed deep, for approximating the input depths of rank join operators in a physical execution plan. At the core of deep lies a general, principled framework that formalizes depth computation in terms of the joint distribution of scores in the base tables. This framework results in a systematic estimation methodology that takes the characteristics of the data directly into account and thus enables more accurate estimates. We develop novel estimation algorithms that provide an efficient realization of the formal deep framework, and describe their integration on top of the statistics module of an existing query optimizer. We validate the performance of deep with an extensive experimental study on data sets of varying characteristics. The results verify the effectiveness of deep as an estimation method and demonstrate its advantages over previously proposed techniques.     

Randomness Buys Depth for Approximate Counting

We show that the promise problem of distinguishing n-bit strings of relative Hamming weight \({1/2 + \Omega(1/{\rm lg}^{d-1} n)}\) from strings of weight \({1/2 - \Omega(1/{\rm \lg}^{d - 1} n)}\) can be solved by explicit, randomized (unbounded fan-in) poly(n)-size depth-d circuits with error \({\leq 1/3}\) , but cannot be solved by deterministic poly(n)-size depth-(d+1) circuits, for every \({d \geq 2}\) ; and the depth of both is tight. Our bounds match Ajtai’s simulation of randomized depth-d circuits by deterministic depth-(d + 2) circuits (Ann. Pure Appl. Logic; ’83) and provide an example where randomization buys resources. To rule out deterministic circuits, we combine Håstad’s switching lemma with an earlier depth-3 lower bound by the author (Computational Complexity 2009). To exhibit randomized circuits, we combine recent analyses by Amano (ICALP ’09) and Brody and Verbin (FOCS ’10) with derandomization. To make these circuits explicit, we construct a new, simple pseudorandom generator that fools tests \({A_1 \times A_2 \times \cdots \times A_{{\rm lg}{n}}}\) for \({A_i \subseteq [n], |A_{i}| = n/2}\) with error 1/n and seed length O(lg n), improving on the seed length \({\Omega({\rm lg}\, n\, {\rm lg}\, {\rm lg}\, n)}\) of previous constructions.