Rendering of forest scenery using 3D textures

Visual simulation of forest scenery is a challenging problem which includes the following tough sub-problems: generation of vegetation, representation of trees, simulation of colour change of leaves, and rendering of numerous trees. Among those sub-problems, this paper treats mainly the last one. A conventional polygon-based rendering algorithm often produces troublesome aliasing effects when it is applied to the objects having complex fine surfaces, such as forest scenery. In this paper, we show that an extended volume rendering technique applied to 3D textures, i.e. volume data in this paper, of trees is effective in the concerned problem. Kajiya left, as further work, the problem of rendering forest scenery by applying his 3D texture called texel. Our rendering method consists of the following three steps: we first generate 3D textures of trees from their polygon-based geometric models, we next arrange the 3D textures, allowing their possible mutual intersections, on the surface of a given polygon-based terrain model according to a simulated vegetation, and we finally produce an image of forest scenery by applying the ray-tracing algorithm including our slightly extended volume rendering technique. © 1997 John Wiley & Sons, Ltd.     

Video registration using dynamic textures

We consider the problem of spatially and temporally registering multiple video sequences of dynamical scenes which contain, but are not limited to, nonrigid objects such as fireworks, flags fluttering in the wind, etc., taken from different vantage points. This problem is extremely challenging due to the presence of complex variations in the appearance of such dynamic scenes. In this paper, we propose a simple algorithm for matching such complex scenes. Our algorithm does not require the cameras to be synchronized, and is not based on frame-by-frame or volume-by-volume registration. Instead, we model each video as the output of a linear dynamical system and transform the task of registering the video sequences to that of registering the parameters of the corresponding dynamical models. As these parameters are not uniquely defined, one cannot directly compare them to perform registration. We resolve these ambiguities by jointly identifying the parameters from multiple video sequences, and converting the identified parameters to a canonical form. This reduces the video registration problem to a multiple image registration problem, which can be efficiently solved using existing image matching techniques. We test our algorithm on a wide variety of challenging video sequences and show that it matches the performance of significantly more computationally expensive existing methods.     

Physically Meaningful Rendering using Tristimulus Colours

In photorealistic image synthesis the radiative transfer equation is often not solved by simulating every wavelength of light, but instead by computing tristimulus transport, for instance using sRGB primaries as a basis. This choice is convenient, because input texture data is usually stored in RGB colour spaces. However, there are problems with this approach which are often overlooked or ignored. By comparing to spectral reference renderings, we show how rendering in tristimulus colour spaces introduces colour shifts in indirect light, violation of energy conservation, and unexpected behaviour in participating media. Furthermore, we introduce a fast method to compute spectra from almost any given XYZ input colour. It creates spectra that match the input colour precisely. Additionally, like in natural reflectance spectra, their energy is smoothly distributed over wide wavelength bands. This method is both useful to upsample RGB input data when spectral transport is used and as an intermediate step for corrected tristimulus‐based transport. Finally, we show how energy conservation can be enforced in RGB by mapping colours to valid reflectances.     

Rendering large scenes using parallel ray tracing

Ray tracing is a powerful technique to generate realistic images of 3D scenes. However, rendering complex scenes may easily exceed the processing and memory capabilities of a single workstation. Distributed processing offers a solution if the algorithm can be parallelized in an efficient way. In this paper a hybrid scheduling approach is presented that combines demand driven and data parallel techniques. Which tasks to process demand driven and which data parallel, is decided by the data intensity of the task and the amount of data locality (coherence) that will be present in the task. By combining demand driven and data driven tasks, a better load balance may be achieved, while at the same time the communication is spread evenly across the network. This leads to a scalable and efficient parallel implementation of the ray tracing algorithm with little restriction on the size of the model data base to be rendered.     

Rendering Soft Shadows using Multilayered Shadow Fins

Generating soft shadows in real time is difficult. Exact methods (such as ray tracing, and multiple light source simulation) are too slow, while approximate methods often overestimate the umbra regions. In this paper, we introduce a new algorithm based on the shadow map method to quickly and highly accurately render soft shadows produced by a light source. Our method builds inner and outer translucent fins on objects to represent the penumbra area inside and outside hard shadows, respectively. The fins are traced into multilayered light space maps to store illuminance adjustment to shadows. The viewing space illuminance buffer is then calculated using those maps. Finally, by blending illuminance and shading, a scene with highly accurate soft shadow effects is produced. Our method does not suffer from umbra overestimation. Physical relations between light, objects and shadows demonstrate the soundness of our approach.     

Stereo for Image-Based Rendering using Image Over-Segmentation

In this paper, we propose a stereo method specifically designed for image-based rendering. For effective image-based rendering, the interpolated views need only be visually plausible. The implication is that the extracted depths do not need to be correct, as long as the recovered views appear to be correct. Our stereo algorithm relies on over-segmenting the source images. Computing match values over entire segments rather than single pixels provides robustness to noise and intensity bias. Color-based segmentation also helps to more precisely delineate object boundaries, which is important for reducing boundary artifacts in synthesized views. The depths of the segments for each image are computed using loopy belief propagation within a Markov Random Field framework. Neighboring MRFs are used for occlusion reasoning and ensuring that neighboring depth maps are consistent. We tested our stereo algorithm on several stereo pairs from the Middlebury data set, and show rendering results based on two of these data sets. We also show results for video-based rendering.     

A unified framework for designing textures using energy optimization

A unified framework is proposed for designing textures using energy optimization and deformation. Our interactive scheme has the ability to globally change the visual properties of texture elements, and locally change texture elements with little user interaction. Given a small sample texture, the design process starts with applying a set of global deformation operations (rotation, translation, mirror, scale and flip) to the sample texture to obtain a set of deformed textures automatically. Then we further make the local deformation to the deformed textures interactively by replacing the local-texture elements regions from other textures. By utilizing the energy optimization method, interactive selections and deformations of local-texture elements are accomplished simply through indicating the positions of texture elements very roughly with a brush tool. Finally the deformed textures are further utilized to create large textures with the fast layer-based texture deformation algorithm, and the wavelet-based energy optimization. Our experimental results demonstrate that the proposed approach can help design a large variety of textures from a small example, change the locations of texture elements, increase or decrease the density of texture elements, and design cyclic marbling textures.     

Associating visual textures with human perceptions using genetic algorithms

This paper deals with an approach allowing to associate visual textures with given human perceptions. Hereby, based on a forward model associating human perceptions for given visual textures, the deduction of an reverse process is presented which is able to associate and characterize visual textures for given human perceptions. For doing so, we propose a constraint-based genetic algorithm approach, which is able to minimize a specific optimization problem containing constraints in form of band-widths for valid individuals (low level features extracted from textures) in a population. The constraints are determined by relationships between (low level) features characterizing textures in form of high-dimensional approximation models. Additionally, in each iteration step checking for valid individuals is carried out with a texture/non-texture classifier or by using a convex hull over a set of valid textures. The whole approach is evaluated based on a real-world texture set used as a start population in the genetic algorithm and by defining various kinds of human perceptions (for which textures are sought) represented by adjective vectors in the aesthetic space. The generated individuals (low level feature vectors) have a high level of fitness (they are quite close to the pre-defined adjective vectors) and a small distance to the initial population. The textures synthesized based on the generated individuals are visualized and compared with textures synthesized by a time-intensive direct texture mixing and re-combination method based on a real-world texture data base.     

边缘和纹理优先的可逆数据隐藏算法

为了提高加水印图像的感知质量,对基于图像像素预测误差的嵌入算法进行了研究。优先选择了图像的边缘和纹理进行差值扩展嵌入,并结合压缩性非常高的溢出位置图,实现了高视觉质量、大嵌入容量的可逆数据隐藏算法。     

Rendering hair using pixel blending and shadow buffers

A technique is described for adding natural-looking hair to standard rendering algorithms. Using an explicit hair model, in which each individual hair is represented by a three-dimensional curve, the technique uses pixel blending combined with Z-buffer and shadow buffer information from the scene to yield a final anti-aliased image with soft shadows. Although developed for rendering human hair, this technique can also be used to render any model consisting of long filaments of sub-pixel width. The technique can be adapted to any rendering method that outputs Z-buffer and shadow buffer information and is amenable to hardware implementation.     

Unsupervised detection and localization of structural textures using projection profiles

The main goal of existing approaches for structural texture analysis has been the identification of repeating texture primitives and their placement patterns in images containing a single type of texture. We describe a novel unsupervised method for simultaneous detection and localization of multiple structural texture areas along with estimates of their orientations and scales in real images. First, multi-scale isotropic filters are used to enhance the potential texton locations. Then, regularity of the textons is quantified in terms of the periodicity of projection profiles of filter responses within sliding windows at multiple orientations. Next, a regularity index is computed for each pixel as the maximum regularity score together with its orientation and scale. Finally, thresholding of this regularity index produces accurate localization of structural textures in images containing different kinds of textures as well as non-textured areas. Experiments using three different data sets show the effectiveness of the proposed method in complex scenes.     

Hierarchical textures

Rich, interesting patterns create an oasis of aesthetic pleasure between the arid lands of complete randomness and complete order. In addition to their inherent beauty, patterns prove useful in graphics when we use them as textures. Textures were originally used to add color detail to shapes. Today, textures control everything from shape details to the placement of extras in digital crowd scenes. Though not appropriate for everything, such patterns can be used for applications from ornamentation to creating complex 3D structures and motion. One easy and powerful way to create rich textures is to start with something simple, and then use that seed along with some rules to “grow” something more complicated. Lots of growing schemes produce both repeating and non-repeating patterns. One of the simplest techniques is called “tiling”, where you place down multiple copies of a single starting shape, or tile, according to some (usually simple) rules. You can create rich tilings in surprisingly many ways. I look at a tiling technique based on using a set of simple symmetry operations     

Audio-visual sports highlights extraction using Coupled Hidden Markov Models

We present our studies on the application of Coupled Hidden Markov Models(CHMMs) to sports highlights extraction from broadcast video using both audio and video information. First, we generate audio labels using audio classification via Gaussian mixture models, and video labels using quantization of the average motion vector magnitudes. Then, we model sports highlights using discrete-observations CHMMs on audio and video labels classified from a large training set of broadcast sports highlights. Our experimental results on unseen golf and soccer content show that CHMMs outperform Hidden Markov Models(HMMs) trained on audio-only or video-only observations. Next, we study how the coupling between the two single-modality HMMs offers improvement on modelling capability by making refinements on the states of the models. We also show that the number of states optimized in this fashion also gives better classification results than other number of states. We conclude that CHMMs provide a promising tool for information fusion techniques in the sports domain for audio-visual event detection and analysis.     

Computer aided generation of stylized maps

Geographic maps have existed from early stages of human civilization. Various styles of visualizing the geographic information have evolved depending on the nature of information and the technology available for visualization. This has led to innumerable map styles. In this work we develop a technique to create maps by combining two‐dimensional and three‐dimensional information such that the resulting maps are both functional and aesthetically appealing. Our technique requires geographical information in vector form and aerial images as inputs. We use computer vision based approaches and user defined inputs to augment the vector data with information that is required to create stylized maps. We define procedural graphics methods to generate a range of geographic elements that can be composed together into a stylized map. We demonstrate our technique by generating example maps of a region in Las Vegas. Copyright © 2007 John Wiley & Sons, Ltd.     

A method for generating pavement textures using the square packing technique

Published online: 2 October 2001     

Monocular Human Pose and Shape Reconstruction using Part Differentiable Rendering

Superior human pose and shape reconstruction from monocular images depends on removing the ambiguities caused by occlusions and shape variance. Recent works succeed in regression-based methods which estimate parametric models directly through a deep neural network supervised by 3D ground truth. However, 3D ground truth is neither in abundance nor can efficiently be obtained. In this paper, we introduce body part segmentation as critical supervision. Part segmentation not only indicates the shape of each body part but helps to infer the occlusions among parts as well. To improve the reconstruction with part segmentation, we propose a part-level differentiable renderer that enables part-based models to be supervised by part segmentation in neural networks or optimization loops. We also introduce a general parametric model engaged in the rendering pipeline as an intermediate representation between skeletons and detailed shapes, which consists of primitive geometries for better interpretability. The proposed approach combines parameter regression, body model optimization, and detailed model registration altogether. Experimental results demonstrate that the proposed method achieves balanced evaluation on pose and shape, and outperforms the state-of-the-art approaches on Human3.6M, UP-3D and LSP datasets.     

遥感图像配准的稳健投影非负矩阵分解方法

由于要配准的目标存在可能的形变,震前和震后遥感图像的配准变得很困难。为了解决这个问题,提出基于稳健的投影非负矩阵分解（RPNMF）的配准方法来精确的配准形变目标。给出一种稳健的投影非负矩阵分解方法来获得震前震后形变目标的共同投影空间,利用在共同投影空间的投影来配准形变目标。为验证该算法的有效性,做了两个实验：2008年5月12日汶川地震前后的SAR图像的配准;唐家山堰塞湖的变化检测。与现有方法进行比较,结果表明该方法能够有效地得到形变目标的共同投影空间,并取得了很好的配准结果;同时,堰塞湖的变化检测也得到了很好的结果。