基于Biharmonic样条插值的图像渐变算法及实现

图像变形是计算机动画的重要技术之一。本文提出一种图像变形方法,利用Biharmonic样条曲面插值实现空间映射,采用后向映射和双线性插值进行图像重采样。在图像变形基础上结合图像融合技术实现了图像渐变。该方法使用灵活,操作者可以简单地改变特征点的数量和位置以控制变形细节而无需其他操作。实验结果表明,该方法产生的过渡图像平滑、自然,每幅变形图像边界稳定且有良好的可视性。     

Q-warping: Direct computation of quadratic reference surfaces

We consider the problem of wrapping around an object, of which two views are available, a reference surface and recovering the resulting parametric flow using direct computations (via spatio-temporal derivatives). The well known examples are affine flow models and eight-parameter flow models-both describing a flow field of a planar reference surface. We extend those classic flow models to deal with a quadric reference surface and work out the explicit parametric form of the flow field. As a result we derive a simple warping algorithm that maps between two views and leaves a residual flow proportional to the 3D deviation of the surface from a virtual quadric surface. The applications include image morphing, model building, image stabilization, and disparate view correspondence     

Polygonal Shape Blending with Topological Evolutions

This paper presents a new general approach to blend 2D shapes with different topologies. All possible topological evolutions are classified into three types by attaching three different topological cells. This formalism is resulted from Morse theory on the behavior of the 3D surface around a non-degenerate critical point. Also we incorporate degenerate topological evolutions into our framework which produce more attractive morphing effects. The user controls the morph by specifying the types of topological evolutions as well as the feature correspondences between the source and target shapes. Some techniques are also provided to control the vertex path during the morphing process. The amount of user input required to produce a morph is directly proportional to the amount of control the user wishes to impose on the process. The user may allow the system to automatically generate the morph as well. Our approaches are totally geometric based and are easy and fast enough in fully interactive time. Many experimental results show the applicability and flexibility of our approaches.     

圆弧路径法，一种新的多边形变形方法

二维形状变形技术在二维角色动画、模式匹配、几何造型、虚拟现实、工业模拟、科学计算可视化等领域有着重要的应用。本文提出了一种顶点路径圆弧法的二维形状变形新方法。该算法通过控制关键帧多边形顶点按照一条特殊的圆弧曲线路径进行运动，实现二维形状变形。通过许多实例的测试表明，该算法效果良好：不仅可以保持首末关键帧形状的共同特征，而且中间插值形状变化自然平滑。同时，我们的方法易于用户交互控制；容易推广到高维情形；计算量较小、能达到系统实时的要求。     

基于控制点的图象变形方法及其应用

根据人脸,特别是根据人脸在人眼变形中对形状和细节的要求,提出了用光滑不等距插值和空间可变线性插值方法来实现基于控制点网格的数字图象变形的想法,其中,光滑不等距插值用来实现图象的坐标变换,而空间可变线性插值用于实现图象的灰度插值。同时,可将变形过程中图象的坐标变换和灰度插值分开操作,用以取代原来行和列的分别变形操作,实验结果证明,用这种方法不仅能获得很好的变形效果,且使用该方法进行人眼的变形操作也取得了满意的结果。     

Illumination Morphing: Generating a Smooth Change of Illumination between Two Color Images

The paper proposes a method for generating a sequence of images with smooth change of illumination from two input images with different lighting conditions. The idea of the proposed method is based on image morphing. While conventional image morphing changes object shapes between two input images, here we focus on changing the illumination between two images. The proposed method uses isoluminance curves as a feature primitive. Isoluminance curves acquired from images are warped based on the correspondence of the curves between two images, and transformed luminance distributions are generated from the warped isoluminance curves. The proposed method called "illumination morphing" is able to generate smooth transition of luminance between two color images. The method does not need even the information about the light sources and 3D object models. The proposed method is a promising technique for many applications requiring a scene with variety of lighting effects, such as movies, TV games, and so on.     

Unsupervised Dense Light Field Reconstruction with Occlusion Awareness

Light field (LF) reconstruction is a fundamental technique in light field imaging and has applications in both software and hardware aspects. This paper presents an unsupervised learning method for LF‐oriented view synthesis, which provides a simple solution for generating quality light fields from a sparse set of views. The method is built on disparity estimation and image warping. Specifically, we first use per‐view disparity as a geometry proxy to warp input views to novel views. Then we compensate the occlusion with a network by a forward‐backward warping process. Cycle‐consistency between different views are explored to enable unsupervised learning and accurate synthesis. The method overcomes the drawbacks of fully supervised learning methods that require large labeled training dataset and epipolar plane image based interpolation methods that do not make full use of geometry consistency in LFs. Experimental results demonstrate that the proposed method can generate high quality views for LF, which outperforms unsupervised approaches and is comparable to fully‐supervised approaches.     

Deep compression of remotely rendered views

Three-dimensional (3-D) models are information-rich and provide compelling visualization effects. However downloading and viewing 3-D scenes over the network may be excessive. In addition low-end devices typically have insufficient power and/or memory to render the scene interactively in real-time. Alternatively,3-D image warping, an image-based-rendering technique that renders a two-dimensional(2-D) depth view to form new views intended from different viewpoints and/or orientations, may be employed on a limited device. In a networked 3-D environment,the warped views may be further compensated by the graphically rendered views and transmitted to clients at times. Depth views can be considered as a compact model of 3-D scenes enabling the remote rendering of complex 3-D environment on relatively low-end devices. The major overhead of the 3-D image warping environment is the transmission of the depth views of the initial and subsequent references. This paper addresses the issue by presenting an effective remote rendering environment based on the deep compression of depth views utilizing the context statistics structure present in depth views. The warped image quality is also explored by reducing the resolution of the depth map. It is shown that proposed deep compression of the remote rendered view significantly outperforms the JPEG2000 and enables the realtime rendering of remote 3-D scene while the degradation of warped image quality is visually imperceptible for the benchmark scenes.     

Interactive Image Morphing on a Single-Chip Multiprocessor using a Multilayered Parallel Image Computing Library

Image morphing uses a combination of warping and image interpolation to produce a gradual visual transformation of one image into another. Due to the complexity of the operations involved in warping, creating the intermediate images that compose a “morph” is computationally intensive. Prior implementations have been slow, taking several seconds or even minutes, to compute each intermediate image. By optimizing the morphing algorithm for use on the Texas Instruments TMS320C80 Multimedia Video Processor (MVP), we have achieved interactive image morphing, which can compute an intermediate image in less than 200 ms. Using the University of Washington Image Computing Library (UWICL) for the MVP, a multilayered parallel image computing library, we were able to implement our interactive morphing algorithm with a minimum amount of programming effort. Thus, in addition to being significantly faster, our implementation is an excellent demonstration of the capability and development potential of the MVP and UWICL combination in higher-level algorithms and applications.     

Tri-view morphing

This paper presents an efficient image-based approach to navigate a scene based on only three wide-baseline uncalibrated images without the explicit use of a 3D model. After automatically recovering corresponding points between each pair of images, an accurate trifocal plane is extracted from the trifocal tensor of these three images. Next, based on a small number of feature marks using a friendly GUI, the correct dense disparity maps are obtained by using our trinocular-stereo algorithm. Employing the barycentric warping scheme with the computed disparity, we can generate an arbitrary novel view within a triangle spanned by three camera centers. Furthermore, after self-calibration of the cameras, 3D objects can be correctly augmented into the virtual environment synthesized by the tri-view morphing algorithm. Three applications of the tri-view morphing algorithm are demonstrated. The first one is 4D video synthesis, which can be used to fill in the gap between a few sparsely located video cameras to synthetically generate a video from a virtual moving camera. This synthetic camera can be used to view the dynamic scene from a novel view instead of the original static camera views. The second application is multiple view morphing, where we can seamlessly fly through the scene over a 2D space constructed by more than three cameras. The last one is dynamic scene synthesis using three still images, where several rigid objects may move in any orientation or direction. After segmenting three reference frames into several layers, the novel views in the dynamic scene can be generated by applying our algorithm. Finally, the experiments are presented to illustrate that a series of photo-realistic virtual views can be generated to fly through a virtual environment covered by several static cameras.     

Adaptive Acquisition of Lumigraphs from Synthetic Scenes

Light fields and Lumigraphs are capable of rendering scenes of arbitrary geometrical or illumination complexity in real time. They are thus interesting ways of interacting with both recorded real-world and high-quality synthetic scenes.
Unfortunately, both light fields and Lumigraph rely on a dense sampling of the illumination to provide a good rendering quality. This induces high costs both in terms of storage requirements and computational resources for the image acquisition. Techniques for acquiring adaptive light field and Lumigraph representations are thus mandatory for practical applications.
In this paper we present a method for the adaptive acquisition of images for Lumigraphs from synthetic scenes. Using image warping to predict the potential improvement in image quality when adding a certain view, we decide which new views of the scene should be rendered and added to the light field. This a-priori error estimator accounts for both visibility problems and illumination effects such as specular highlights.     

Minimal Warping: Planning Incremental Novel‐view Synthesis

Observing that many visual effects (depth‐of‐field, motion blur, soft shadows, spectral effects) and several sampling modalities (time, stereo or light fields) can be expressed as a sum of many pinhole camera images, we suggest a novel efficient image synthesis framework that exploits coherency among those images. We introduce the notion of “distribution flow” that represents the 2D image deformation in response to changes in the high‐dimensional time‐, lens‐, area light‐, spectral‐, etc. coordinates. Our approach plans the optimal traversal of the distribution space of all required pinhole images, such that starting from one representative root image, which is incrementally changed (warped) in a minimal fashion, pixels move at most by one pixel, if at all. The incremental warping allows extremely simple warping code, typically requiring half a millisecond on an Nvidia Geforce GTX 980Ti GPU per pinhole image. We show, how the bounded sampling does introduce very little errors in comparison to re‐rendering or a common warping‐based solution. Our approach allows efficient previews for arbitrary combinations of distribution effects and imaging modalities with little noise and high visual fidelity.     

可调度的小波网格变形

三维网格模型的Morphing是计算机动画研究的重要内容,利用频谱工具控制变形成份出现的次序是对变形路径进行控制的有效方法之一。但在三维网格模型上无法直接进行频谱分析,使得该技术一直无法在网格模型的Morphing中发挥作用。论文在数字几何处理框架的基础上,提出了利用小波分析进行可控网格变形的新方法。针对变形研究和数字几何处理中的关键问题——特征点对齐和采样,分别提出了基于Harmonic映射的参数化网格warping的特征对齐方法以及新型的自适应采样方法。实验表明,该方法稳定可靠,计算量适中,变形结果更为丰富。     

多特征融合的图像自动变形

目的 图像变形算法中特征基元提取和匹配方式大部分都是采用人机交互的方式进行,并且在遮挡区域变形时出现较多的鬼影和模糊现象,使得针对同一场景图像变形实现繁琐且效果不佳,针对这些问题提出一种基于多特征融合的自动图像变形算法。方法 该算法提取多种图像特征信息（如Surf特征算子、Harris算子、Canny算子等）并进行多特征融合匹配,得到一个分布适当且对应关系正确的三角网格,再结合图像变形,实现自动图像插值。结果 实验结果显示,自动的提取特征基元有效地减少了人工操作,而多特征融合匹配有效地抑制了图像变形时边缘或遮挡区域鬼影的产生。结论 提出的融合匹配方法,将不同的特征信息有效地融合匹配从而改善了图像变形算法。通过对多组实验结果进行问卷调查,91%的参与者认为该算法有效地改进图像变形结果。     

一种街景全景生成的改进算法

当前主流的全景拼接软件在进行投影变换的过程中对街景影像上匹配点对的精度的依赖性较强。为了增强其鲁棒性,采取在投影后的影像上获取匹配点对并将其反投影回街景影像坐标系的策略来减少对初始匹配点对的依赖。此外,针对投影后相邻图像之间的错位导致全景拼接产生缝隙的问题,采用多层B样条插值算法对相邻图像的光流进行插值来消除影像间的错位。实验结果表明,上述的算法策略提高了投影转换的精度,明显改善了全景拼接的效果。     

基于小波的自动图像变形

图像变形算法中最费时的是在源图像和目标图像中确定变形特征基元；而到目前为止，大部分图像变形算法都是通过人工或半人工的方式来实现的．如果两幅源图像间的特征基元的确定能自动实现，则不但可以减轻用户的负担，而且可以极大地节省变形算法所需的时间，同时提高了算法的效率．提出了一种基于小波变换的自动图像变形方法．该方法完全依赖于给定的两幅源图像，自动生成期间的变形图像，不需要预先在两幅源图像中指定特征基元．实验结果证明，用本文的算法不仅能自动完成变形操作，而且也取得了满意的结果．     

基于DIBR和图像修复的任意视点绘制

基于深度的图像绘制（DIBR）是高级视频应用的关键技术,为提高视点变换的图像质量,提出一种基于DIBR和图像修复的任意视点绘制方法。首先对深度图像进行形态学处理,以减少视点变换产生的空洞,平滑目标视点内部的物体轮廓;利用视点变换方程生成目标视点;对含有空洞的目标视点采用图像修复算法进行后处理,设计了含有深度项的代价函数,在深度的约束下进行纹理搜索,将最佳匹配块填补到空洞;在修复空洞的过程中采用亮度优先的策略以适应不同的色度采样格式。实验的主观效果对比和PSNR数据都显示本文算法比其他算法更为优越。     

生成当前视点目标图像的快速逆映射算法

利用极线的属性及深度图像隐含的边界信息,提出了一种可以处理非深度连续图像的快速逆映射算法,以从多幅参考图像精确合成当前视点目标图像.该算法分为3步:首先由参考图像的深度信息得到图像中的边界;然后从参考图像中选出一幅作为主参考图像,利用极线的整体匹配特性和对应点在极线上分布的单调性,逐个处理目标极线,以生成目标图像;最后根据其他参考图像填补目标图像中的空洞.由于在第2步中只需处理参考图像的边界点对即可得到对应目标极线上所有点的深度信息及其在参考极线上的对应点,因此,新算法很好地提高了速度.在填补空洞时,利用参考图像的边界信息及所隐含的遮挡关系,还提出了一些加速的方法.     

Real-time exploration and photorealistic reconstruction of large natural environments

This paper presents a hybrid (geometry- and image-based) framework suitable for providing photorealistic walkthroughs of large, complex outdoor scenes, based only on a small set of real images from the scene. To this end, a novel data representation of a 3D scene is proposed, which is called morphable 3D panoramas. Motion is assumed to be taking place along a predefined path of the 3D environment and the input to the system is a sparse set of stereoscopic views at certain positions (key positions) along that path (one view per position). An approximate local 3D model is constructed from each view, capable of capturing the photometric and geometric properties of the scene only locally. Then, during the rendering process, a continuous morphing (both photometric as well as geometric) takes place between successive local 3D models, using what we call a ‘morphable 3D model’. For the estimation of the photometric morphing, a robust algorithm capable of extracting a dense field of 2D correspondences between wide-baseline images is used, whereas, for the geometric morphing, a novel method of computing 3D correspondences between local models is proposed. In this way, a physically valid morphing is always produced, which is thus kept transparent from the user. Moreover, a highly optimized rendering path is used during morphing. Thanks to the use of appropriate pixel and vertex shaders, this rendering path can be run fully in 3D graphics hardware and thus allows for high frame rates. Our system can be extended to handle multiple stereoscopic views (and therefore multiple local models) per key position of the path (related by a camera rotation). In this case, one local 3D panorama (per key position) is constructed, comprising all local 3D models therein, and so a ‘morphable 3D panorama’ is now used during the rendering process. For handling the geometric consistency of each 3D panorama, a technique which is based on solving a partial differential equation is adopted. The effectiveness of our framework is demonstrated by using it for the 3D visual reconstruction of the Samaria Gorge in Crete.     

Quality assessment of 3D synthesized images based on structural and textural distortion

Depth image based rendering (DIBR) is a popular technique for rendering virtual 3D views in stereoscopic and autostereoscopic displays. The quality of DIBR-synthesized images may decrease due to various factors, e.g., imprecise depth maps, poor rendering techniques, inaccurate camera parameters. The quality of synthesized images is important as it directly affects the overall user experience. Therefore, the need arises for designing algorithms to estimate the quality of the DIBR-synthesized images. The existing 2D image quality assessment metrics are found to be insufficient for 3D view quality estimation because the 3D views not only contain color information but also make use of disparity to achieve the real depth sensation. In this paper, we present a new algorithm for evaluating the quality of DIBR generated images in the absence of the original references. The human visual system is sensitive to structural information; any deg radation in structure or edges affects the visual quality of the image and is easily noticeable for humans. In the proposed metric, we estimate the quality of the synthesized view by capturing the structural and textural distortion in the warped view. The structural and textural information from the input and the synthesized images is estimated and used to calculate the image quality. The performance of the proposed quality metric is evaluated on the IRCCyN IVC DIBR images dataset. Experimental evaluations show that the proposed metric outperforms the existing 2D and 3D image quality metrics by achieving a high correlation with the subjective ratings.