Bidirectional Texture Function Modeling: A State of the Art Survey

An ever-growing number of real-world computer vision applications require classification, segmentation, retrieval, or realistic rendering of genuine materials. However, the appearance of real materials dramatically changes with illumination and viewing variations. Thus, the only reliable representation of material visual properties requires capturing of its reflectance in as wide range of light and camera position combinations as possible. This is a principle of the recent most advanced texture representation, the bidirectional texture function (BTF). Multispectral BTF is a seven-dimensional function that depends on view and illumination directions as well as on planar texture coordinates. BTF is typically obtained by measurement of thousands of images covering many combinations of illumination and viewing angles. However, the large size of such measurements has prohibited their practical exploitation in any sensible application until recently. During the last few years, the first BTF measurement, compression, modeling, and rendering methods have emerged. In this paper, we categorize, critically survey, and psychophysically compare such approaches, which were published in this newly arising and important computer vision and graphics area.     

Perceptually Motivated BRDF Comparison using Single Image

Surface reflectance of real‐world materials is now widely represented by the bidirectional reflectance distribution function (BRDF) and also by spatially varying representations such as SVBRDF and the bidirectional texture function (BTF). The raw surface reflectance measurements are typically compressed or fitted by analytical models, that always introduce a certain loss of accuracy. For its evaluation we need a distance function between a reference surface reflectance and its approximate version. Although some of the past techniques tried to reflect the perceptual sensitivity of human vision, they have neither optimized illumination and viewing conditions nor surface shape. In this paper, we suggest a new image‐based methodology for comparing different anisotropic BRDFs. We use optimization techniques to generate a novel surface which has extensive coverage of incoming and outgoing light directions, while preserving its features and frequencies that are important for material appearance judgments. A single rendered image of such a surface along with simultaneously optimized lighting and viewing directions leads to the computation of a meaningful BRDF difference, by means of standard image difference predictors. A psychophysical experiments revealed that our surface provides richer information on material properties than the standard surfaces often used in computer graphics, e.g., sphere or blob.     

Decorating surfaces with bidirectional texture functions

We present a system for decorating arbitrary surfaces with bidirectional texture functions (BTF). Our system generates BTFs in two steps. First, we automatically synthesize a BTF over the target surface from a given BTF sample. Then, we let the user interactively paint BTF patches onto the surface such that the painted patches seamlessly integrate with the background patterns. Our system is based on a patch-based texture synthesis approach known as quilting. We present a graphcut algorithm for BTF synthesis on surfaces and the algorithm works well for a wide variety of BTF samples, including those which present problems for existing algorithms. We also describe a graphcut texture painting algorithm for creating new surface imperfections (e.g., dirt, cracks, scratches) from existing imperfections found in input BTF samples. Using these algorithms, we can decorate surfaces with real-world textures that have spatially-variant reflectance, fine-scale geometry details, and surfaces imperfections. A particularly attractive feature of BTF painting is that it allows us to capture imperfections of real materials and paint them onto geometry models. We demonstrate the effectiveness of our system with examples.     

Synthesis and rendering of bidirectional texture functions on arbitrary surfaces

The bidirectional texture function (BTF) is a 6D function that describes the appearance of a real-world surface as a function of lighting and viewing directions. The BTF can model the fine-scale shadows, occlusions, and specularities caused by surface mesostructures. In this paper, we present algorithms for efficient synthesis of BTFs on arbitrary surfaces and for hardware-accelerated rendering. For both synthesis and rendering, a main challenge is handling the large amount of data in a BTF sample. To addresses this challenge, we approximate the BTF sample by a small number of 4D point appearance functions (PAFs) multiplied by 2D geometry maps. The geometry maps and PAFs lead to efficient synthesis and fast rendering of BTFs on arbitrary surfaces. For synthesis, a surface BTF can be generated by applying a texton-based sysnthesis algorithm to a small set of 2D geometry maps while leaving the companion 4D PAFs untouched. As for rendering, a surface BTF synthesized using geometry maps is well-suited for leveraging the programmable vertex and pixel shaders on the graphics hardware. We present a real-time BTF rendering algorithm that runs at the speed of about 30 frames/second on a mid-level PC with an ATI Radeon 8500 graphics card. We demonstrate the effectiveness of our synthesis and rendering algorithms using both real and synthetic BTF samples.     

Bidirectional Texture Contrast Function

Three dimensional surface corrugations on globally smooth surfaces give rise to brightness modulations of global shading patterns. We study systematic variations of such 3D image texture as a function of illumination and viewing geometry. The 3D texture is especially noticeable near the shadow terminator (for collimated illumination) or near the dark pole (for hemispherical diffuse illumination). We find that a simple micro-facet model, assuming locally Lambertian scattering, suffices to robustly describe texture contrast gradients of a large variety (measured and rendered textures; laboratory and field conditions) in a semi-quantitative manner. Robust statistical measures of the texture allows one to draw inferences concerning the nature of the light field (collimated to diffuse) and of surface roughness parameters, which can be used as input to the simplest BRDF models.     

Capture and fusion of 3d surface texture

Image fusion is a process that multiple images of a scene are combined to form a single image. The aim of image fusion is to preserve the full content and retain important features of each original image. In this paper, we propose a novel approach based on wavelet transform to capture and fusion of real-world rough surface textures, which are commonly used in multimedia applications and referred to as3D surface texture. These textures are different from 2D textures as their appearances can vary dramatically with different illumination conditions due to complex surface geometry and reflectance properties. In our approach, we first extract gradient/height and albedo maps from sample 3D surface texture images as their representation. Then we measure saliency of wavelet coefficients of these 3D surface texture representations. The saliency values reflect the meaningful content of the wavelet coefficients and are consistent with human visual perception. Finally we fuse the gradient/height and albedo maps based on the measured saliency values. This novel scheme aims to preserve the original texture patterns together with geometry and reflectance characteristics from input images. Experimental results show that the proposed approach can not only capture and fuse 3D surface texture under arbitrary illumination directions, but also has the ability to retain the surface geometry properties and preserve perceptual features in the original images.     

利用采样球体提取真实感纹理

提出了一种从真实物体中提取纹理的方法.利用具有复杂纹理的参考球体作为被采样物体,计算其组成材质的BRDF(bidirectional reflectance distribution function)模型参数以及各点由不同材质构成的比例,形成一幅材质权重图.该图作为纹理映射到3D物体上后,配合BRDF模型参数进行渲染,形成一种适用于重光照(relighting)的纹理.被渲染物体可根据自身方位以及光源亮度/方位呈现出自然的光影变化,达到较为逼真的外观效果.     

Extreme compression and modeling of bidirectional texture function

The recent advanced representation for realistic real-world materials in virtual reality applications is the Bidirectional Texture Function (BTF) which describes rough texture appearance for varying illumination and viewing conditions. Such a function can be represented by thousands of measurements (images) per material sample. The resulting BTF size excludes its direct rendering in graphical applications and some compression of these huge BTF data spaces is obviously inevitable. In this paper we present a novel, fast probabilistic model-based algorithm for realistic BTF modeling allowing an extreme compression with the possibility of a fast hardware implementation. Its ultimate aim is to create a visual impression of the same material without a pixel-wise correspondence to the original measurements. The analytical step of the algorithm starts with a BTF space segmentation and a range map estimation by photometric stereo of the BTF surface, followed by the spectral and spatial factorization of selected sub-space color texture images. Single mono-spectral band-limited factors are independently modeled by their dedicated spatial probabilistic model. During rendering, the sub-space images of arbitrary size are synthesized and both color (possibly multi-spectral) and range information is combined in a bump-mapping filter according to the view and illumination directions. The presented model offers a huge BTF compression ratio unattainable by any alternative sampling-based BTF synthesis method. Simultaneously this model can be used to reconstruct missing parts of the BTF measurement space.     

Representing and Recognizing the Visual Appearance of Materials using Three-dimensional Textons

We study the recognition of surfaces made from different materials such as concrete, rug, marble, or leather on the basis of their textural appearance. Such natural textures arise from spatial variation of two surface attributes: (1) reflectance and (2) surface normal. In this paper, we provide a unified model to address both these aspects of natural texture. The main idea is to construct a vocabulary of prototype tiny surface patches with associated local geometric and photometric properties. We call these 3D textons. Examples might be ridges, grooves, spots or stripes or combinations thereof. Associated with each texton is an appearance vector, which characterizes the local irradiance distribution, represented as a set of linear Gaussian derivative filter outputs, under different lighting and viewing conditions.Given a large collection of images of different materials, a clustering approach is used to acquire a small (on the order of 100) 3D texton vocabulary. Given a few (1 to 4) images of any material, it can be characterized using these textons. We demonstrate the application of this representation for recognition of the material viewed under novel lighting and viewing conditions. We also illustrate how the 3D texton model can be used to predict the appearance of materials under novel conditions.     

Increasing the Spatial Resolution of BTF Measurement with Scheimpflug Imaging

We present an improved way of acquiring spatially varying surface reflectance represented by a bidirectional texture function (BTF). Planar BTF samples are measured as images at several inclination angles which puts constraints on the minimum depth of field of cameras used in the measurement instrument. For standard perspective imaging, we show that the size of a sample measured and the achievable spatial resolution are strongly interdependent and limited by diffraction at the lens' aperture. We provide a formula for this relationship. We overcome the issue of the required depth of field by using Scheimpflug imaging further enhanced by an anamorphic attachment. The proposed optics doubles the spatial resolution of images compared to standard perspective imaging optics. We built an instrument prototype with the proposed optics that is portable and allows for measurement on site. We show rendered images using the visual appearance measured by the instrument prototype.     

Bidirectional Texture Function Compression Based on Multi-Level Vector Quantization

The Bidirectional Texture Function (BTF) is becoming widely used for accurate representation of real-world material appearance. In this paper a novel BTF compression model is proposed. The model resamples input BTF data into a parametrization, allowing decomposition of individual view and illumination dependent texels into a set of multi-dimensional conditional probability density functions. These functions are compressed in turn using a novel multi-level vector quantization algorithm. The result of this algorithm is a set of index and scale code-books for individual dimensions. BTF reconstruction from the model is then based on fast chained indexing into the nested stored code-books. In the proposed model, luminance and chromaticity are treated separately to achieve further compression. The proposed model achieves low distortion and compression ratios 1:233–1:2040, depending on BTF sample variability. These results compare well with several other BTF compression methods with predefined compression ratios, usually smaller than 1:200. We carried out a psychophysical experiment comparing our method with LPCA method. BTF synthesis from the model was implemented on a standard GPU, yielded interactive framerates. The proposed method allows the fast importance sampling required by eye-path tracing algorithms in image synthesis.     

Capturing and rendering geometry details for BTF-mapped surfaces

Bidirectional texture functions, or BTFs, accurately model reflectance variation at a fine (meso-) scale as a function of lighting and viewing direction. BTFs also capture view-dependent visibility variation, also called masking or parallax, but only within surface contours. Mesostructure detail is neglected at silhouettes, so BTF-mapped objects retain the coarse shape of the underlying model. We augment BTF rendering to obtain approximate mesoscale silhouettes. Our new representation, the 4D mesostructure distance function (MDF), tabulates the displacement from a reference frame where a ray first intersects the mesoscale geometry beneath as a function of ray direction and ray position along that reference plane. Given an MDF, the mesostructure silhouette can be rendered with a per-pixel depth peeling process on graphics hardware, while shading and local parallax are handled by the BTF. Our approach allows real-time rendering, handles complex, non-height-field mesostructure, requires that no additional geometry be sent to the rasterizer other than the mesh triangles, is more compact than textured visibility representations used previously, and, for the first time, can be easily measured from physical samples. We also adapt the algorithm to capture detailed shadows cast both by and onto BTF-mapped surfaces. We demonstrate the efficiency of our algorithm on a variety of BTF data, including real data acquired using our BTF–MDF measurement system.     

Texture histograms as a function of irradiation and viewing direction

The textural appearance of materials encountered in our daily environment depends on two directions, the irradiation and viewing direction. We investigate the bidirectional grey level histograms of a large set of materials, obtained from a texture database. We distinguish important categories, relate the various effects to physical mechanisms, and list material attributes that influence the bidirectional histograms. We use a model for rough surfaces with locally diffuse and/or specular reflection properties, a class of materials that commonly occurs, to generate bidirectional histograms and obtain close agreement with experimental data. We discuss several applications of bidirectional texture functions and histograms. In particular, we present a new approach to texture mapping based on bidirectional histograms. For 3D texture, this technique is superior to standard 2D texture mapping at hardly any extra computational cost or memory requirements.     

Tileable BTF

This paper presents a modular framework to efficiently apply the bidirectional texture functions (BTF) onto object surfaces. The basic building blocks are the BTF tiles. By constructing one set of BTF tiles, a wide variety of objects can be textured seamlessly without resynthesizing the BTF. The proposed framework nicely decouples the surface appearance from the geometry. With this appearance-geometry decoupling, one can build a library of BTF tile sets to instantaneously dress and render various objects under variable lighting and viewing conditions. The core of our framework is a novel method for synthesizing seamless high-dimensional BTF tiles, which are difficult for existing synthesis techniques. Its key is to shorten the cutting paths and broaden the choices of samples so as to increase the chance of synthesizing seamless BTF tiles. To tackle the enormous data, the tile synthesis process is performed in a compressed domain. This not only allows the handling of large BTF data during the synthesis, but also facilitates the compact storage of the BTF in a GPU memory during the rendering.     

辐射度场景中双向纹理函数表面的绘制

双向纹理函数(BTF)表面一般采用点采样数据来定义表面的光照属性,因而这类表面很难运用基于面片分割的辐射度方法进行绘制,提出一种将辐射度算法扩展到包括BTF表面场景的有效方法.对表面的BTF样本区域首先进行像素聚类,再在各个像素类内对视线采样方向做进一步自适应的聚类,在各个视线类内像素分别拟合一个低频光照甬数,并求它们在各个视线类内光照细节的高频光照函数.低频光照函数作为该表面区域的平均反射属性参与辐射度计算,生成场景的整体光照效果;然后利用计算的辐射度值和高频光照函数重建该表面区域的BTF材质细节.文中方法不仅取得了较高的压缩效率,而且在BTF材质表面产生了辉映等全局光照效果.最后利用硬件实现了视点快速改变时的场景绘制.     

The analysis and recognition of real-world textures in threedimensions

The observed image texture for a rough surface has a complex dependence on the illumination and viewing angles due to effects such as foreshortening, local shading, interreflections, and the shadowing and occlusion of surface elements. We introduce the dimensionality surface as a representation for the visual complexity of a material sample. The dimensionality surface defines the number of basis textures that are required to represent the observed textures for a sample as a function of ranges of illumination and viewing angles. Basis textures are represented using multiband correlation functions that consider both within and between color band correlations. We examine properties of the dimensionality surface for real materials using the Columbia Utrecht Reflectance and Texture (CUReT) database. The analysis shows that the dependence of the dimensionality surface on ranges of illumination and viewing angles is approximately linear with a slope that depends on the complexity of the sample. We extend the analysis to consider the problem of recognizing rough surfaces in color images obtained under unknown illumination and viewing geometry. We show, using a set of 12,505 images from 61 material samples, that the information captured by the multiband correlation model allows surfaces to be recognized over a wide range of conditions. We also show that the use of color information provides significant advantages for three-dimensional texture recognition     

Edit Propagation on Bidirectional Texture Functions

We propose an efficient method for editing bidirectional texture functions (BTFs) based on edit propagation scheme. In our approach, users specify sparse edits on a certain slice of BTF. An edit propagation scheme is then applied to propagate edits to the whole BTF data. The consistency of the BTF data is maintained by propagating similar edits to points with similar underlying geometry/reflectance. For this purpose, we propose to use view independent features including normals and reflectance features reconstructed from each view to guide the propagation process. We also propose an adaptive sampling scheme for speeding up the propagation process. Since our method needn't any accurate geometry and reflectance information, it allows users to edit complex BTFs with interactive feedback.     

Capture and Synthesis of 3D Surface Texture

We present and compare five approaches for capturing, synthesising and relighting real 3D surface textures. Unlike 2D texture synthesis techniques they allow the captured textures to be relit using illumination conditions that differ from those of the original. We adapted a texture quilting method due to Efros and combined this with five different relighting representations, comprising: a set of three photometric images; surface gradient and albedo maps; polynomial texture maps; and two eigen based representations using 3 and 6 base images.We used twelve real textures to perform quantitative tests on the relighting methods in isolation. We developed a qualitative test for the assessment of the complete synthesis systems. Ten observers were asked to rank the images obtained from the five methods using five real textures. Statistical tests were applied to the rankings.The six-base-image eigen method produced the best quantitative relighting results and in particular was better able to cope with specular surfaces. However, in the qualitative tests there were no significant performance differences detected between it and the other two top performers. Our conclusion is therefore that the cheaper gradient and three-base-image eigen methods should be used in preference, especially where the surfaces are Lambertian or near Lambertian.