基于网格优化的图像纹理替换方法

提出一种新的图像纹理替换方法,该方法在原始图像精确几何及光照信息未知的情况下,用新的纹理有效替换原始图像目标区域的纹理并逼真地保持了原始纹理的扭曲变形和光照效果.该方法借助反求的法向信息,通过一种基于网格边长伸展的参数化方法优化求得原始图像目标区域上划分的网格在纹理空间中的对应网格,有效模拟了原始纹理由于隐藏几何和透视投影引起的纹理扭曲效果;进而通过YCbCr颜色空间转换和传递保持了原始纹理的光照阴影效果.由于求取纹理采样规则的优化方法仅仅与原始图像空间相关,而与替换纹理无关,因而对目标图像处理一次后,可以有效地应用于任意新纹理的替换.通过适当的交互,替换纹理的纹元尺度渐变及纹理自遮挡效果亦可有效模拟.实验展示了良好的纹理替换效果.     

Fast multi-scale joint bilateral texture upsampling

We present a new approach using a multi-scale joint bilateral filter for upsampling the synthesized texture generated by optimization-based methods. Our method is based on the following motivation: if the available exemplar texture is used as a priority to upsample the synthesized texture, a high resolution result that prevents image blurring can be obtained. Our multi-scale joint bilateral upsampling applies a spatial filter on each multi-scale layer of the synthesized texture, and jointly applies a similar range filter on exemplar texture, which guides the interpolation from low to high resolution, by magnifying and combining the upsampled information; the details of the upsampled texture are progressively enhanced, and the image blurring artifacts can be effectively avoided. We offer an accelerated joint bilateral filter, which enables our upsampling method to interactively generate a large texture. In addition, we propose a detail-aware texture optimization approach that incorporates image detail in texture optimization to improve the quality of the synthesized texture, on which the multi-scale joint bilateral filter works to generate a more convincing result. We show results for upsampling image and video textures and compare them to traditional upsampling methods, by this demonstrating that with low computational and memory costs, our method achieves better results.     

Global Texture Mapping for Dynamic Objects

We propose a novel framework to generate a global texture atlas for a deforming geometry. Our approach distinguishes from prior arts in two aspects. First, instead of generating a texture map for each timestamp to color a dynamic scene, our framework reconstructs a global texture atlas that can be consistently mapped to a deforming object. Second, our approach is based on a single RGB‐D camera, without the need of a multiple‐camera setup surrounding a scene. In our framework, the input is a 3D template model with an RGB‐D image sequence, and geometric warping fields are found using a state‐of‐the‐art non‐rigid registration method [GXW*15] to align the template mesh to noisy and incomplete input depth images. With these warping fields, our multi‐scale approach for texture coordinate optimization generates a sharp and clear texture atlas that is consistent with multiple color observations over time. Our approach is accelerated by graphical hardware and provides a handy configuration to capture a dynamic geometry along with a clean texture atlas. We demonstrate our approach with practical scenarios, particularly human performance capture. We also show that our approach is resilient on misalignment issues caused by imperfect estimation of warping fields and inaccurate camera parameters.     

Texture mixing and texture movie synthesis using statisticallearning

We present an algorithm based on statistical learning for synthesizing static and time-varying textures matching the appearance of an input texture. Our algorithm is general and automatic and it works well on various types of textures, including 1D sound textures, 2D texture images, and 3D texture movies. The same method is also used to generate 2D texture mixtures that simultaneously capture the appearance of a number of different input textures. In our approach, input textures are treated as sample signals generated by a stochastic process. We first construct a tree representing a hierarchical multiscale transform of the signal using wavelets. From this tree, new random trees are generated by learning and sampling the conditional probabilities of the paths in the original tree. Transformation of these random trees back into signals results in new random textures. In the case of 2D texture synthesis, our algorithm produces results that are generally as good as or better than those produced by previously described methods in this field. For texture mixtures, our results are better and more general than those produced by earlier methods. For texture movies, we present the first algorithm that is able to automatically generate movie clips of dynamic phenomena such as waterfalls, fire flames, a school of jellyfish, a crowd of people, etc. Our results indicate that the proposed technique is effective and robust     

一种基于网格参数化的图像适应方法

图像适应是指将高分辨率的数字图像显示在手机、PDA等屏幕较小的显示终端上的过程.提出一种全新的基于网格参数化的图像适应方法,该方法的关键在于把图像表示为特征网格,从而将图像适应问题转化为网格的参数化,即求取一个与该特征网格同拓扑,且具有目标屏幕尺寸的网格.为了突出图像中的重要物体,该方法建立了源图像对应的特征网格与图像显著度的关系;通过优化基于显著度伸长的网格参数化的能量来求解适应图像的网格;然后借助纹理映射生成适应图像.另外,该方法在参数化的过程中增加了对显著区域和背景结构的约束,能够在保持并增强图像中重要物体的同时,使适应图像的结构不发生明显形变.该方法能够方便地处理具有复杂背景和包含多目标物体的图像的适应问题.实验结果显示了该方法的有效性.     

A unified framework for designing textures using energy optimization

A unified framework is proposed for designing textures using energy optimization and deformation. Our interactive scheme has the ability to globally change the visual properties of texture elements, and locally change texture elements with little user interaction. Given a small sample texture, the design process starts with applying a set of global deformation operations (rotation, translation, mirror, scale and flip) to the sample texture to obtain a set of deformed textures automatically. Then we further make the local deformation to the deformed textures interactively by replacing the local-texture elements regions from other textures. By utilizing the energy optimization method, interactive selections and deformations of local-texture elements are accomplished simply through indicating the positions of texture elements very roughly with a brush tool. Finally the deformed textures are further utilized to create large textures with the fast layer-based texture deformation algorithm, and the wavelet-based energy optimization. Our experimental results demonstrate that the proposed approach can help design a large variety of textures from a small example, change the locations of texture elements, increase or decrease the density of texture elements, and design cyclic marbling textures.     

Texture Synthesis From Photographs

The goal of texture synthesis is to generate an arbitrarily large high‐quality texture from a small input sample. Generally, it is assumed that the input image is given as a flat, square piece of texture, thus it has to be carefully prepared from a picture taken under ideal conditions. Instead we would like to extract the input texture from any surface from within an arbitrary photograph. This introduces several challenges: Only parts of the photograph are covered with the texture of interest, perspective and scene geometry introduce distortions, and the texture is non‐uniformly sampled during the capture process. This breaks many of the assumptions used for synthesis. In this paper we combine a simple novel user interface with a generic per‐pixel synthesis algorithm to achieve high‐quality synthesis from a photograph. Our interface lets the user locally describe the geometry supporting the textures by combining rational Bézier patches. These are particularly well suited to describe curved surfaces under projection. Further, we extend per‐pixel synthesis to account for arbitrary texture sparsity and distortion, both in the input image and in the synthesis output. Applications range from synthesizing textures directly from photographs to high‐quality texture completion.     

Video retargeting combining warping and summarizing optimization

We construct a unified interactive video retargeting system for video summarization, completion, and reshuffling. Our system combines the advantages of both video warping and summarizing processing. We first warp the video to present initial editing results, then refine the results using patch-based summarizing optimization, which mainly eliminates possible distortion produced in the warping step. We develop a Mean Value Coordinate (MVC) warping method due to its simplicity and efficiency used in the initialization. For refining processing, the summarization optimization is built on a 3D bidirectional similarity measure between the original and edited video, to preserve the coherence and completeness of the final editing result. We further improve the quality of summarization by applying a color histogram matching during the optimization, and accelerate the summarization optimization by using a constrained 3D Patch-Match algorithm. Experiment results show that the proposed video retargeting system effectively supports video summarization, completion, and reshuffling while avoiding issues like texture broken, video jittering, and detail losing.     

Novel approaches for synthesizing video textures

Video texture, a novel type of medium, can produce a new video with a continuously varying stream of images from a recorded video. A classic approach to generate video textures is to apply principal components analysis (PCA) for dimensionality reduction (i.e. extraction of frame signatures) and autoregressive (AR) process for prediction purposes. In this paper we investigate the use of other dimensionality reduction techniques to generate accurate video textures. Based on our experiments, the quality of video textures can be improved further. We also propose a new approach for generating video textures using probabilistic principal components analysis (PPCA) and Gaussian process dynamical model (GPDM) to synthesize video textures which contain frames that never appeared before and with similar motions as original videos. Furthermore, we propose two ways of generating online video textures by applying the incremental Isomap and incremental Spatio-temporal Isomap (IST-Isomap). Both approaches can produce good online video texture results. In particular, IST-Isomap, that we propose, is more suitable for sparse video data (e.g. cartoon).     

Multiscaled Texture Synthesis Using Multisized Pixel Neighborhoods

Most recent efforts in texture synthesis have focused on using local statistics - that is, selecting and stitching the textures from an input texture sample on the basis of a local color match to enforce textural features either pixel by pixel or patch by patch. A novel texture synthesis method produces high-quality results by introducing a multiscaled texture similarity measurement. Compared with other multiscaled methods, this approach focuses on measuring texture properties at different scales ranging from local to global using an adaptive similarity metric that accounts for texture variations across different image regions     

Texturing fluids

We present a novel technique for synthesizing textures over dynamically changing fluid surfaces. We use both image textures as well as bump maps as example inputs. Image textures can enhance the rendering of the fluid by either imparting realistic appearance to it or by stylizing it, whereas bump maps enable the generation of complex micro-structures on the surface of the fluid that may be very difficult to synthesize using simulation. To generate temporally coherent textures over a fluid sequence, we transport texture information, i.e. color and local orientation, between free surfaces of the fluid from one time step to the next. This is accomplished by extending the texture information from the first fluid surface to the 3D fluid domain, advecting this information within the fluid domain along the fluid velocity field for one time step, and interpolating it back onto the second surface -- this operation, in part, uses a novel vector advection technique for transporting orientation vectors. We then refine the transported texture by performing texture synthesis over the second surface using our "surface texture optimization" algorithm, which keeps the synthesized texture visually similar to the input texture and temporally coherent with the transported one. We demonstrate our novel algorithm for texture synthesis on dynamically evolving fluid surfaces in several challenging scenarios.     

A novel image decomposition approach and its applications

The current state-of-the-art edge-preserving decomposition techniques may not be able to fully separate textures while preserving edges. This may generate artifacts in some applications, e.g., edge detection, texture transfer, etc. To solve this problem, a novel image decomposition approach based on explicit texture separation from large scale components of an image is presented. We first apply a Gaussian structure-texture decomposition, to separate the majority of textures out of the input image. However, residual textures are still visible around the strong edges. To remove these residuals, an asymmetric sampling operator is proposed and followed by a joint bilateral correction to remove an excessive blur effect. We demonstrate that our approach is well suited for the tasks such as texture transfer, edge detection, non-photorealistic rendering, and tone mapping. The results show our approach outperforms existing state-of-the-art image decomposition approaches.     

Feature-aware natural texture synthesis

This article presents a framework for natural texture synthesis and processing. This framework is motivated by the observation that given examples captured in natural scene, texture synthesis addresses a critical problem, namely, that synthesis quality can be affected adversely if the texture elements in an example display spatially varied patterns, such as perspective distortion, the composition of different sub-textures, and variations in global color pattern as a result of complex illumination. This issue is common in natural textures and is a fundamental challenge for previously developed methods. Thus, we address it from a feature point of view and propose a feature-aware approach to synthesize natural textures. The synthesis process is guided by a feature map that represents the visual characteristics of the input texture. Moreover, we present a novel adaptive initialization algorithm that can effectively avoid the repeat and verbatim copying artifacts. Our approach improves texture synthesis in many images that cannot be handled effectively with traditional technologies.     

Feature-Based Deformable Surface Detection with Self-Occlusion Reasoning

This paper presents a method for detecting a textured deformed surface in an image. It uses (wide-baseline) point matches between a template and the input image. The main contribution of the paper is twofold. First, we propose a robust method based on local surface smoothness capable of discarding outliers from the set of point matches. Our method handles large proportions of outliers (beyond 70% with less than 15% of false positives) even when the surface self-occludes. Second, we propose a method to estimate a self-occlusion resistant warp from point matches. Our method allows us to realistically retexture the input image. A pixel-based (direct) registration approach is also proposed. Bootstrapped by our robust point-based method, it finely tunes the warp parameters using the value (intensity or color) of all the visible surface pixels. The proposed framework was tested with simulated and real data. Convincing results are shown for the detection and retexturing of deformed surfaces in challenging images.     

人群仿真中角色纹理多样化方法

人群仿真中角色模型的多样化能营造出符合实际的效果。通过模型纹理处理技术来加强人群仿真中的人物外观多样化,是解决仿真场景中三维模型外观差异性的有效手段。提出一种基于模板的纹理多样化生成方法。首先给不带纹理坐标的网格模型展UV后作为纹理模板;然后利用基于图像分割方法处理UV展开图得到纹理的衣饰语义信息;在此基础上实现纹理图像的纯色着色和图案填充,从而生成各种不同的纹理,实现人群仿真的角色多样化。所提方法实用可行,较好地解决了虚拟角色制作效率问题,快速生成视觉上合理的大规模人群。     

Aura 3D textures

This paper presents a new technique, called aura 3D textures, for generating solid textures based on input examples. Our method is fully automatic and requires no user interactions in the process. Given an input texture sample, our method first creates its aura matrix representations and then generates a solid texture by sampling the aura matrices of the input sample constrained in multiple view directions. Once the solid texture is generated, any given object can be textured by the solid texture. We evaluate the results of our method based on extensive user studies. Based on the evaluation results using human subjects, we conclude that our algorithm can generate faithful results of both stochastic and structural textures with an average successful rate of 76.4 percent. Our experimental results also show that the new method outperforms Wei and Levoy's method and is comparable to that proposed by Jagnow et al. (2004)     

飞行器表面矢量场数据动态纹理可视化方法

面向飞行器表面流场数据可视化的应用需求,提出一种基于线性卷积（LIC）及纹理平流（IBFVS）相结合的动态纹理可视化方法。算法通过将IBFVS方法的背景随机噪声替换为LIC纹理方式,结合了LIC纹理结果对比度高及IBFVS方法生成速度快的优势;针对LIC绘制速度慢的不足,利用GPU对曲面矢量场投影并插值,生成规则矢量数据场;用GPU对LIC部分进行并行加速,有效提高了LIC纹理图像产生速度;将LIC结果图像加入到IBFVS进行平流,生成纹理图像,最后加入颜色映射,丰富流场信息。实验结果表明,该方法生成的飞行器表面动态纹理图像对比度高,清晰度强,实时绘制性能好。     

图像和视频的低多边形渲染

低多边形是近来艺术设计界的热门风格.为了提高图像和视频低多边形风格化的质量,提出一种基于边缘特征和超像素分割的图像和视频低多边形渲染方法.首先提取相邻超像素的交点以及对特征边和超像素边界的差集的均匀采样点作为三角网格顶点,并执行Delaunay三角剖分来生成初始三角网格;然后采用带约束的二次误差度量方法对生成的网格进行...     

Deformation-based interactive texture design using energy optimization

In this paper, we present a novel interactive texture design scheme based on deformation and energy optimization. Given a small sample texture, the design process starts with applying a set of deformation operations to the sample texture to obtain a set of deformed textures. Then local changes to those deformed textures are further made by replacing their local regions with the texture elements interactively selected from other textures. Such a deform–select–replace process is iterated many times until the desired deformed textures are obtained. Finally the deformed textures are composed to form a large texture with graph-cut optimization. By combining the graph-cut algorithm with an energy optimization process, interactive selections of local texture elements are done simply through indicating the positions of texture elements very roughly with a brush tool. Our experimental results demonstrate that the proposed technique can be used for designing a large variety of versatile textures from a single small sample texture, increasing or decreasing the density of texture elements, as well as for synthesizing textures from multiple sources.