Selecting texture resolution using a task‐specific visibility metric

In real‐time rendering, the appearance of scenes is greatly affected by the quality and resolution of the textures used for image synthesis. At the same time, the size of textures determines the performance and the memory requirements of rendering. As a result, finding the optimal texture resolution is critical, but also a non‐trivial task since the visibility of texture imperfections depends on underlying geometry, illumination, interactions between several texture maps, and viewing positions. Ideally, we would like to automate the task with a visibility metric, which could predict the optimal texture resolution. To maximize the performance of such a metric, it should be trained on a given task. This, however, requires sufficient user data which is often difficult to obtain. To address this problem, we develop a procedure for training an image visibility metric for a specific task while reducing the effort required to collect new data. The procedure involves generating a large dataset using an existing visibility metric followed by refining that dataset with the help of an efficient perceptual experiment. Then, such a refined dataset is used to retune the metric. This way, we augment sparse perceptual data to a large number of per‐pixel annotated visibility maps which serve as the training data for application‐specific visibility metrics. While our approach is general and can be potentially applied for different image distortions, we demonstrate an application in a game‐engine where we optimize the resolution of various textures, such as albedo and normal maps.     

Texture Particles

This paper presents an analytical extension of texture synthesis techniques based on the distribution of elementary texture components. Our approach is similar to the bombing, cellular, macrostructured and lapped textures techniques, but provides the user with more control on both the texture analysis and synthesis phases. Therefore, high quality results can be obtained for a large number of structured or stochastic textures (bricks, marble, lawn, etc.). The analysis consists in decomposing textures into elementary components — that we call ``texture particles'' — and for which we analyze their specific spatial arrangements. The synthesis then consists in recomposing similar textures directly on arbitrary surfaces by taking into account the previously computed arrangements, extended to 3D surfaces. Compared to ``pixel‐based'' analysis and synthesis methods, which have been recently generalized to arbitrary surfaces, our approach has three major advantages: (1) it is fast, which allows the user to interactively control the synthesis process. This further allows us to propose a large number of tools, granting a high degree of artistic freedom to the user. (2) It avoids the visual deterioration of the texture components by preserving their shapes as well as their spatial arrangements. (3) The texture particles can be not only images, but also 3D geometric elements, which extends significantly the domain of application.     

Decorating surfaces with bidirectional texture functions

We present a system for decorating arbitrary surfaces with bidirectional texture functions (BTF). Our system generates BTFs in two steps. First, we automatically synthesize a BTF over the target surface from a given BTF sample. Then, we let the user interactively paint BTF patches onto the surface such that the painted patches seamlessly integrate with the background patterns. Our system is based on a patch-based texture synthesis approach known as quilting. We present a graphcut algorithm for BTF synthesis on surfaces and the algorithm works well for a wide variety of BTF samples, including those which present problems for existing algorithms. We also describe a graphcut texture painting algorithm for creating new surface imperfections (e.g., dirt, cracks, scratches) from existing imperfections found in input BTF samples. Using these algorithms, we can decorate surfaces with real-world textures that have spatially-variant reflectance, fine-scale geometry details, and surfaces imperfections. A particularly attractive feature of BTF painting is that it allows us to capture imperfections of real materials and paint them onto geometry models. We demonstrate the effectiveness of our system with examples.     

A Randomized Approach for Patch-based Texture Synthesis using Wavelets

We present a wavelet‐based approach for selecting patches in patch‐based texture synthesis. We randomly select the first block that satisfies a minimum error criterion, computed from the wavelet coefficients (using 1D or 2D wavelets) for the overlapping region. We show that our wavelet‐based approach improves texture synthesis for samples where previous work fails, mainly textures with prominent aligned features. Also, it generates similar quality textures when compared against texture synthesis using feature maps with the advantage that our proposed method uses implicit edge information (since it is embedded in the wavelet coefficients) whereas feature maps rely explicitly on edge features. In previous work, the best patches are selected among all possible using a L2 norm on the RGB or grayscale pixel values of boundary zones. The L2 metric provides the raw pixel‐to‐pixel difference, disregarding relevant image structures — such as edges — that are relevant in the human visual system and therefore on synthesis of new textures.     

Extending natural textures with multi-scale synthesis

This paper presents a texture synthesis algorithm that was designed for the tile-less generation of large images of arbitrary size from small sample images. The synthesised texture shows features that are visually similar to the sample over a wide frequency range. The development of the algorithm aimed at achieving high quality results for a large range of natural textures, incorporation of the original samples in the synthesis product, ease of use and good texturing speed even with input sample data two magnitudes larger than used by previous techniques. Like other algorithms we utilise an implicit texture model by copying arbitrary shaped texture patches from the sample to the destination over a multi-scale image pyramid. Our method combines the advantages of different previous techniques with respect to quality. A mixture of exhaustive searching, massive parallel computing and the well-known LBG-algorithm ensures a good balance between texturing quality and speed.     

可控的连续多尺度纹理合成

针对同一幅合成图像的不同区域间,如何实现纹理图案的多尺度连续变化,提出一种基于像素的多尺度纹理合成算法。在逐像素点合成的基础上,通过为已知匹配点建立一个到连续的多分辨率样图的正向映射,并重新采样合成像素点,实现合成结果的任意尺度连续渐变的可控性。实验证明该方法不增加大量的处理时间,并可适用于大多数基于样图的逐点合成方法。     

Real-time structured texture synthesis and editing using image-mesh analogies

We present a novel texture synthesis technique designed to reproduce at real-time frame-rates example texture images, with a special focus on patterns characterized by structural arrangements. Unlike current pixel-, patch- or texton-based schemes, that operate in image space, our approach is structural. We propose to assimilate texture images to corresponding 2D geometric meshes (called texture meshes). Our analysis mainly consists in generating automatically these meshes, while synthesis is then based on the creation of new vertex/polygon distributions matching some arrangement map. The output texture image is obtained by rasterizing the previously generated polygons using graphics hardware capabilities, which guarantees high speed performance. By operating in geometry space instead of image/pixel space, the proposed structural approach has a major advantage over current techniques: beyond pure texture reproduction, it allows us to define various tools, which allow users to further modify locally or globally and in real-time structural components of textures. By controlling the arrangement map, users can substitute new meshes in order to completely modify the structural appearance of input textures, yet maintaining a certain visual resemblance with the initial example image.     

Peek‐in‐the‐Pic: Flying Through Architectural Scenes From a Single Image*

Many casually taken ‘tourist’ photographs comprise of architectural objects like houses, buildings, etc. Reconstructing such 3D scenes captured in a single photograph is a very challenging problem. We propose a novel approach to reconstruct such architectural scenes with minimal and simple user interaction, with the goal of providing 3D navigational capability to an image rather than acquiring accurate geometric detail. Our system, Peek‐in‐the‐Pic, is based on a sketch‐based geometry reconstruction paradigm. Given an image, the user simply traces out objects from it. Our system regards these as perspective line drawings, automatically completes them and reconstructs geometry from them. We make basic assumptions about the structure of traced objects and provide simple gestures for placing additional constraints. We also provide a simple sketching tool to progressively complete parts of the reconstructed buildings that are not visible in the image and cannot be automatically completed. Finally, we fill holes created in the original image when reconstructed buildings are removed from it, by automatic texture synthesis. Users can spend more time using interactive texture synthesis for further refining the image. Thus, instead of looking at flat images, a user can fly through them after some simple processing. Minimal manual work, ease of use and interactivity are the salient features of our approach.     

Interactive texture design and synthesis from mesh sketches

Geometry mesh introduces user control into texture synthesis and editing, and brings more variations in the synthesized results. But still two problems related remain in need of better solutions. One problem is generating the meshes with desired size and pattern efficiently from easier user inputs. The other problem is improving the quality of synthesized results with mesh information. We present a new two-step texture design and synthesis method that addresses these two problems. Besides example texture, a small piece of mesh sketch drawn by hand or detected from example texture is input to our algorithm. And then a mesh synthesis method of geometry space is provided to avoid optimizations cell by cell. Distance and orientation features are introduced to improve the quality of mesh rasterization. Results show that with our method, users can design and synthesize textures from mesh sketches easily and interactively.     

Multiscaled Texture Synthesis Using Multisized Pixel Neighborhoods

Most recent efforts in texture synthesis have focused on using local statistics - that is, selecting and stitching the textures from an input texture sample on the basis of a local color match to enforce textural features either pixel by pixel or patch by patch. A novel texture synthesis method produces high-quality results by introducing a multiscaled texture similarity measurement. Compared with other multiscaled methods, this approach focuses on measuring texture properties at different scales ranging from local to global using an adaptive similarity metric that accounts for texture variations across different image regions     

基于块的任意曲面上的纹理合成

一种新的曲面纹理合成技术被提出,它采用求最小割集来解决块拼接时的礁缝走样问题。算法在预处理期对模型的三维网格进行了重建,使其规则并且建立网格点与参数化栅格采样后的图像点一一对应的关系,无论对合成的速度和质量都有很大的提高。提出了在曲面上进行纹理粘贴的算法,对于渲染存在不完整性的自然物体取得了较好的效果。     

Capture and fusion of 3d surface texture

Image fusion is a process that multiple images of a scene are combined to form a single image. The aim of image fusion is to preserve the full content and retain important features of each original image. In this paper, we propose a novel approach based on wavelet transform to capture and fusion of real-world rough surface textures, which are commonly used in multimedia applications and referred to as3D surface texture. These textures are different from 2D textures as their appearances can vary dramatically with different illumination conditions due to complex surface geometry and reflectance properties. In our approach, we first extract gradient/height and albedo maps from sample 3D surface texture images as their representation. Then we measure saliency of wavelet coefficients of these 3D surface texture representations. The saliency values reflect the meaningful content of the wavelet coefficients and are consistent with human visual perception. Finally we fuse the gradient/height and albedo maps based on the measured saliency values. This novel scheme aims to preserve the original texture patterns together with geometry and reflectance characteristics from input images. Experimental results show that the proposed approach can not only capture and fuse 3D surface texture under arbitrary illumination directions, but also has the ability to retain the surface geometry properties and preserve perceptual features in the original images.     

Patch‐based Texture Interpolation

In this paper, we present a novel exemplar‐based technique for the interpolation between two textures that combines patch‐based and statistical approaches. Motivated by the notion of texture as a largely local phenomenon, we warp and blend small image neighborhoods prior to patch‐based texture synthesis. In addition, interpolating and enforcing characteristic image statistics faithfully handles high frequency detail. We are able to create both intermediate textures as well as continuous transitions. In contrast to previous techniques computing a global morphing transformation on the entire input exemplar images, our localized and patch‐based approach allows us to successfully interpolate between textures with considerable differences in feature topology for which no smooth global warping field exists.     

A hybrid-based texture synthesis approach

Texture synthesis has been an active research area in computer graphics, vision, and image processing. A hybrid texture synthesis approach that combines the strength of Ashikhmins [1] and Liangs [11] algorithms is presented in this paper. Using patches from the input sample texture to extend the input itself, more suitable valid candidate pixels in the input sample can be more precisely determined. An extra original position array is used to record the valid pixel position in the extended region of input sample texture such that the candidate pixel can be retrieved efficiently. Pasting patches with more global information of texture features to initialize the output texture, the texture synthesis process tends to grow larger pieces in the output texture so that the perceptual similarity is reserved as much as possible. Many classes of textures have been used to test our approach. As the experimental results showed, the proposed hybrid approach effectively and efficiently synthesizes high-quality textures from a wide variety of textures.     

Texture Synthesis using Exact Neighborhood Matching

In this paper we present an elegant pixel‐based texture synthesis technique that is able to generate visually pleasing results from source textures of both stochastic and structured nature. Inspired by the observation that the most common artifacts that occur when synthesizing textures are high‐frequency discontinuities, our technique tries to avoid these artifacts by forcing at least one of the direct neighboring pixels in each causal neighborhood to match within a predetermined threshold. This does not only avoid deterioration of the visual quality, but also results in faster synthesis timings. We demonstrate our technique on a variety of stochastic and structured textures.     

Multi-exemplar inhomogeneous texture synthesis

This paper presents a new framework for the semi-automatic synthesis of an inhomogeneous texture from a theoretically infinite number of input exemplars. Our algorithm generates a result with natural multiway transitions, maintaining a reasonable synthesis time to keep interactivity. We introduce intermediate textures between multiple exemplars, called ‘transition textures’ which enable our algorithm to produce smooth transitions between input exemplar patches. We also propose ‘image index correction’ to achieve a synthesis speed that is independent of the number of input exemplars by smart placement of transition-texture samples. Our algorithm is semiautomatic, spatially deterministic and well suited to acceleration via parallel processors. Also, our algorithm can easily control the synthesis result to design a texture visual with user interactions.     

An Optimal Control Approach for Texture Metamorphosis

In this paper, we introduce a new texture metamorphosis approach for interpolating texture samples from a source texture into a target texture. We use a new energy optimization scheme derived from optimal control principles which exploits the structure of the metamorphosis optimality conditions. Our approach considers the change in pixel position and pixel appearance in a single framework. In contrast to previous techniques that compute a global warping based on feature masks of textures, our approach allows to transform one texture into another by considering both intensity values and structural features of textures simultaneously. We demonstrate the usefulness of our approach for different textures, such as stochastic, semi‐structural and regular textures, with different levels of complexities. Our method produces visually appealing transformation sequences with no user interaction.     

Texture Stationarization: Turning Photos into Tileable Textures

Texture synthesis has grown into a mature field in computer graphics, allowing the synthesis of naturalistic textures and images from photographic exemplars. Surprisingly little work, however, has been dedicated to synthesizing tileable textures, that is, textures that when laid out in a regular grid of tiles form a homogeneous appearance suitable for use in memory‐sensitive real‐time graphics applications. One of the key challenges in doing so is that most natural input exemplars exhibit uneven spatial variations that, when tiled, show as repetitive patterns. We propose an approach to synthesize tileable textures while enforcing stationarity properties that effectively mask repetitions while maintaining the unique characteristics of the exemplar. We explore a number of alternative measures for texture stationarity and show how each measure can be integrated into a standard texture synthesis method (PatchMatch) to enforce stationarity at user‐controlled scales. We demonstrate the efficacy of our approach using a database of 118 exemplar images, both from publicly available sources as well as new ones captured under uncontrolled conditions, and we quantitatively analyze alternative stationarity measures for their robustness across many test runs using different random seeds. In conclusion, we suggest a novel synthesis approach that employs local histogram matching to reliably turn input photographs of natural surfaces into tiles well suited for artifact‐free tiling.     

A Subjective Evaluation of Texture Synthesis Methods

This paper presents the results of a user study which quantifies the relative and absolute quality of example‐based texture synthesis algorithms. In order to allow such evaluation, a list of texture properties is compiled, and a minimal representative set of textures is selected to cover these. Six texture synthesis methods are compared against each other and a reference on a selection of twelve textures by non‐expert participants (N = 67). Results demonstrate certain algorithms successfully solve the problem of texture synthesis for certain textures, but there are no satisfactory results for other types of texture properties. The presented textures and results make it possible for future work to be subjectively compared, thus facilitating the development of future texture synthesis methods.