Video texture modelling and synthesis using fractal processes

The authors propose a new approach for the synthesis of natural video textures using a fractal- based approach. Specifically, a video texture is modelled according to the three-dimensional (3D) extended self-similar (ESS) model introduced, which generalises the fractional Brownian motion process. The analysis of original video textures is based on the estimation of the autocorrelation functions (ACFs) of the textures' increments. The 3D-ESS model is then used to synthesise a process whose increments have the same ACFs of the given prototype. The synthesis is accomplished by generalising to the 3D case the incremental Fourier synthesis algorithm. Experimental results for the analysis and synthesis of natural video textures are eventually provided.     

Dimensionality Reduction on Multi-Dimensional Transfer Functions for Multi-Channel Volume Data Sets

The design of transfer functions for volume rendering is a non-trivial task. This is particularly true for multi-channel data sets, where multiple data values exist for each voxel, which requires multi-dimensional transfer functions. In this paper, we propose a new method for multi-dimensional transfer function design. Our new method provides a framework to combine multiple computational approaches and pushes the boundary of gradient-based multi-dimensional transfer functions to multiple channels, while keeping the dimensionality of transfer functions at a manageable level, i.e., a maximum of three dimensions, which can be displayed visually in a straightforward way. Our approach utilizes channel intensity, gradient, curvature and texture properties of each voxel. Applying recently developed nonlinear dimensionality reduction algorithms reduces the high-dimensional data of the domain. In this paper, we use Isomap and Locally Linear Embedding as well as a traditional algorithm, Principle Component Analysis. Our results show that these dimensionality reduction algorithms significantly improve the transfer function design process without compromising visualization accuracy. We demonstrate the effectiveness of our new dimensionality reduction algorithms with two volumetric confocal microscopy data sets.     

Mesh-guided optimized retexturing for image and video

This paper presents an approach of replacing textures of specified regions in the input image and video using stretch-based mesh optimization.. The retexturing results have the similar distortion and shading effects conforming to the underlying geometry and lighting conditions. For replacing textures in single image,two important steps are developed: the stretch-based mesh parametrization incorporating the recovered normal information is deduced to imitate perspective distortion of the region of interest; the Poisson-based refinement process is exploited to account for texture distortion at fine scale.The luminance of the input image is preserved through color transfer in YCbCr color space. Our approach is independent of the replaced textures. Once the input image is processed, any new texture can be applied to efficiently generate the retexturing results. For video retexturing, we propose key-frame-based texture replacement extended and generalized from the image retexturing. Our approach repeatedly propagates the replacement result of key frame to the rest of the frames. We develop the local motion optimization scheme to deal with the inaccuracies and errors of robust optical flow when tracking moving objects. Visibility shifting and texture drifting are effectively alleviated using graphcut segmentation algorithm and the global optimization to smooth trajectories of the tracked points over temporal domain. Our experimental results showed that the proposed approach can generate visually pleasing results for both image and video.     

Adaptive learning codebook for action recognition

Learning a compact and yet discriminative codebook is an important procedure for local feature-based action recognition. A common procedure involves two independent phases: reducing the dimensionality of local features and then performing clustering. Since the two phases are disconnected, dimensionality reduction does not necessarily capture the dimensions that are greatly helpful for codebook creation. What’s more, some dimensionality reduction techniques such as the principal component analysis do not take class separability into account and thus may not help build an effective codebook. In this paper, we propose the weighted adaptive metric learning (WAML) which integrates the two independent phases into a unified optimization framework. This framework enables to select indispensable and crucial dimensions for building a discriminative codebook. The dimensionality reduction phase in the WAML is optimized for class separability and adaptively adjusts the distance metric to improve the separability of data. In addition, the video word weighting is smoothly incorporated into the WAML to accurately generate video words. Experimental results demonstrate that our approach builds a highly discriminative codebook and achieves comparable results to other state-of-the-art approaches.     

Visualization of graphical data in a user-specified 2D space using a weighted Isomap method

We present a visualization system for exploring the high-dimensional graphical data, such as textures or 3D models, in 2D space using the dimensionality reduction method. To arrange high-dimensional data in a meaningful 2D space, we develop a novel semi-supervised dimensionality reduction method that can embed data of high dimension in a user-defined 2D coordinate system that is meaningful in terms of the properties of the data. This is achieved by modifying the Isomap method by weighting the data so that the resulting coordinates have no degeneracies and are orthogonal.     

A model validation approach to texture recognition and inpainting

A new approach to texture recognition and inpainting problems is proposed. The approach is based on the robust model validation and state estimation techniques. The proposed solutions require the modeling of textures by using uncertain dynamical systems. We propose a new modeling method which is efficient in terms of computational and memory requirements. The main aspects of the modeling method include system identification and order reduction of marginally stable uncertain discrete-time systems. To demonstrate the results, both static-image textures and video textures (also known as dynamic textures) are considered.     

基于监督双限制连接Isomap算法的带钢表面缺陷图像分类方法

根据带钢表面缺陷图像具有复杂纹理结构、包含大量干扰信息、具备高维非线性几何结构等特点,本文提出基于监督双限制连接Isomap方法的带钢表面缺陷图像降维方法（dls-Isomap）.该方法以Isomap降维方法为基础,对其邻域图的连接方式进行K邻域（K-nearest neighbor,KNN）和ε-半径两个方面的限制性连接,并使用数据类别作为监督对类间邻域点进行扩展连接.针对多类Roll-swiss数据实验表明,dls-Isomap降维方法不仅能够在低维空间中完整嵌入所有数据点,而且能保持数据各类内和类间的几何结构,以及解决Isomap算法存在的“短路边”问题;针对带钢表面缺陷图像分类实验表明,基于dls-Isomap的新分类方法适合含水、油渍等干扰较多的带钢表面缺陷的分类任务,其中冷轧带钢5类缺陷识别率可以达78%.含水渍的热轧带钢缺陷识别率可以达到93%,其中水渍干扰图像的识别率达到97.6%.     

基于有监督增量式等距离映射的人脸识别

针对等距离映射（Isomap）算法无法对后续采集的测试样本单独进行降维处理和未能利用样本点分类信息的不足,提出了一种有监督的增量式等距离映射算法（SIIsomap）,并采取小波变换对图像进行预处理。通过对ORL数据库实验证明,SIIsomap算法与Isomap算法相比大大降低了处理新增样本点的计算时间,并且提高了识别精度。     

Fast multi-scale joint bilateral texture upsampling

We present a new approach using a multi-scale joint bilateral filter for upsampling the synthesized texture generated by optimization-based methods. Our method is based on the following motivation: if the available exemplar texture is used as a priority to upsample the synthesized texture, a high resolution result that prevents image blurring can be obtained. Our multi-scale joint bilateral upsampling applies a spatial filter on each multi-scale layer of the synthesized texture, and jointly applies a similar range filter on exemplar texture, which guides the interpolation from low to high resolution, by magnifying and combining the upsampled information; the details of the upsampled texture are progressively enhanced, and the image blurring artifacts can be effectively avoided. We offer an accelerated joint bilateral filter, which enables our upsampling method to interactively generate a large texture. In addition, we propose a detail-aware texture optimization approach that incorporates image detail in texture optimization to improve the quality of the synthesized texture, on which the multi-scale joint bilateral filter works to generate a more convincing result. We show results for upsampling image and video textures and compare them to traditional upsampling methods, by this demonstrating that with low computational and memory costs, our method achieves better results.     

基于CCIPCA和ICA降维的文本分类研究

文本分类中采用向量空间模型来表达文本特征,维数巨大,关键是对高维的特征集进行降维处理,而一般的分解算法无法处理大规模的高维问题。采用CCIPCA与ICA相结合的特征提取方法可以有效地实现文本特征降维。实验结果表明降维提高了分类器的效率和效果。     

基于独立分量分析的高光谱图像降维与压缩算法

论文提出了一种基于快速独立分量分析的高光谱图像降维算法.利用虚拟维数算法估计需要保留的独立分量数目,采用非监督端元提取算法自动获取端元矢量,并对快速独立分量分析的混合矩阵进行有效初始化.采用最大噪声分离变换对原始数据进行预处理,利用快速独立分量分析从变换后的主分量中依次提取出各端元对应的独立分量,最后对各个独立分量分别实施无损压缩.实验结果表明,该算法降维后的独立分量具有较好的地物分类性能,并且可以获得较好的压缩性能.     

Dimensionality reduction-based spoken emotion recognition

To improve effectively the performance on spoken emotion recognition, it is needed to perform nonlinear dimensionality reduction for speech data lying on a nonlinear manifold embedded in a high-dimensional acoustic space. In this paper, a new supervised manifold learning algorithm for nonlinear dimensionality reduction, called modified supervised locally linear embedding algorithm (MSLLE) is proposed for spoken emotion recognition. MSLLE aims at enlarging the interclass distance while shrinking the intraclass distance in an effort to promote the discriminating power and generalization ability of low-dimensional embedded data representations. To compare the performance of MSLLE, not only three unsupervised dimensionality reduction methods, i.e., principal component analysis (PCA), locally linear embedding (LLE) and isometric mapping (Isomap), but also five supervised dimensionality reduction methods, i.e., linear discriminant analysis (LDA), supervised locally linear embedding (SLLE), local Fisher discriminant analysis (LFDA), neighborhood component analysis (NCA) and maximally collapsing metric learning (MCML), are used to perform dimensionality reduction on spoken emotion recognition tasks. Experimental results on two emotional speech databases, i.e. the spontaneous Chinese database and the acted Berlin database, confirm the validity and promising performance of the proposed method.     

An effective double-bounded tree-connected Isomap algorithm for microarray data classification

Isometric mapping (Isomap) is a popular nonlinear dimensionality reduction technique which has shown high potential in visualization and classification. However, it appears sensitive to noise or scarcity of observations. This inadequacy may hinder its application for the classification of microarray data, in which the expression levels of thousands of genes in a few normal and tumor sample tissues are measured. In this paper we propose a double-bounded tree-connected variant of Isomap, aimed at being more robust to noise and outliers when used for classification and also computationally more efficient. It differs from the original Isomap in the way the neighborhood graph is generated: in the first stage we apply a double-bounding rule that confines the search to at most k nearest neighbors contained within an ε-radius hypersphere; the resulting subgraphs are then joined by computing a minimum spanning tree among the connected components. We therefore achieve a connected graph without unnaturally inflating the values of k and ε. The computational experiences show that the new method performs significantly better in terms of accuracy with respect to Isomap, k-edge-connected Isomap and the direct application of support vector machines to data in the input space, consistently across seven microarray datasets considered in our tests.     

Generating High-quality Superpixels in Textured Images

Superpixel segmentation is important for promoting various image processing tasks. However, existing methods still have difficulties in generating high-quality superpixels in textured images, because they cannot separate textures from structures well. Though texture filtering can be adopted for smoothing textures before superpixel segmentation, the filtering would also smooth the object boundaries, and thus weaken the quality of generated superpixels. In this paper, we propose to use the adaptive scale box smoothing instead of the texture filtering to obtain more high-quality texture and boundary information. Based on this, we design a novel distance metric to measure the distance between different pixels, which considers boundary, color and Euclidean distance simultaneously. As a result, our method can achieve high-quality superpixel segmentation in textured images without texture filtering. The experimental results demonstrate the superiority of our method over existing methods, even the learning-based methods. Benefited from using boundaries to guide superpixel segmentation, our method can also suppress noise to generate high-quality superpixels in non-textured images.     

MesoGAN: Generative Neural Reflectance Shells

We introduce MesoGAN, a model for generative 3D neural textures. This new graphics primitive represents mesoscale appearance by combining the strengths of generative adversarial networks (StyleGAN) and volumetric neural field rendering. The primitive can be applied to surfaces as a neural reflectance shell; a thin volumetric layer above the surface with appearance parameters defined by a neural network. To construct the neural shell, we first generate a 2D feature texture using StyleGAN with carefully randomized Fourier features to support arbitrarily sized textures without repeating artefacts. We augment the 2D feature texture with a learned height feature, which aids the neural field renderer in producing volumetric parameters from the 2D texture. To facilitate filtering, and to enable end-to-end training within memory constraints of current hardware, we utilize a hierarchical texturing approach and train our model on multi-scale synthetic datasets of 3D mesoscale structures. We propose one possible approach for conditioning MesoGAN on artistic parameters (e.g. fibre length, density of strands, lighting direction) and demonstrate and discuss integration into physically based renderers.     

Texture Particles

This paper presents an analytical extension of texture synthesis techniques based on the distribution of elementary texture components. Our approach is similar to the bombing, cellular, macrostructured and lapped textures techniques, but provides the user with more control on both the texture analysis and synthesis phases. Therefore, high quality results can be obtained for a large number of structured or stochastic textures (bricks, marble, lawn, etc.). The analysis consists in decomposing textures into elementary components — that we call ``texture particles'' — and for which we analyze their specific spatial arrangements. The synthesis then consists in recomposing similar textures directly on arbitrary surfaces by taking into account the previously computed arrangements, extended to 3D surfaces. Compared to ``pixel‐based'' analysis and synthesis methods, which have been recently generalized to arbitrary surfaces, our approach has three major advantages: (1) it is fast, which allows the user to interactively control the synthesis process. This further allows us to propose a large number of tools, granting a high degree of artistic freedom to the user. (2) It avoids the visual deterioration of the texture components by preserving their shapes as well as their spatial arrangements. (3) The texture particles can be not only images, but also 3D geometric elements, which extends significantly the domain of application.     

Texturing of layered surfaces for optimal viewing

This paper is a contribution to the literature on perceptually optimal visualizations of layered three-dimensional surfaces. Specifically, we develop guidelines for generating texture patterns, which, when tiled on two overlapped surfaces, minimize confusion in depth-discrimination and maximize the ability to localize distinct features. We design a parameterized texture space and explore this texture space using a "human in the loop" experimental approach. Subjects are asked to rate their ability to identify Gaussian bumps on both upper and lower surfaces of noisy terrain fields. Their ratings direct a genetic algorithm, which selectively searches the texture parameter space to find fruitful areas. Data collected from these experiments are analyzed to determine what combinations of parameters work well and to develop texture generation guidelines. Data analysis methods include ANOVA, linear discriminant analysis, decision trees, and parallel coordinates. To confirm the guidelines, we conduct a post-analysis experiment, where subjects rate textures following our guidelines against textures violating the guidelines. Across all subjects, textures following the guidelines consistently produce high rated textures on an absolute scale, and are rated higher than those that did not follow the guidelines.     

Manifold learning based speaker dependent dimension reduction for robust text independent speaker verification

Speaker verification has been studied widely from different points of view, including accuracy, robustness and being real-time. Recent studies have turned toward better feature stability and robustness. In this paper we study the effect of nonlinear manifold based dimensionality reduction for feature robustness. Manifold learning is a popular recent approach for nonlinear dimensionality reduction. Algorithms for this task are based on the idea that each data point may be described as a function of only a few parameters. Manifold learning algorithms attempt to uncover these parameters in order to find a low-dimensional representation of the data. From the manifold based dimension reduction approaches, we applied the widely used Isometric mapping (Isomap) algorithm. Since in the problem of speaker verification, the input utterance is compared with the model of the claiming client, a speaker dependent feature transformation would be beneficial for deciding on the identity of the speaker. Therefore, our first contribution is to use Isomap dimension reduction approach in the speaker dependent context and compare its performance with two other widely used approaches, namely principle component analysis and factor analysis. The other contribution of our work is to perform the nonlinear transformation in a speaker-dependent framework. We evaluated this approach in a GMM based speaker verification framework using Tfarsdat Telephone speech dataset for different noises and SNRs and the evaluations have shown reliability and robustness even in low SNRs. The results also show better performance for the proposed Isomap approach compared to the other approaches.     

Robust CoHOG feature extraction in human-centered image/video management system

Many human-centered image and video management systems depend on robust human detection. To extract robust features for human detection, this paper investigates the following shortcomings of co-occurrence histograms of oriented gradients (CoHOGs) which significantly limit its advantages: 1) The magnitudes of the gradients are discarded, and only the orientations are used; 2) the gradients are not smoothed, and thus, aliasing effect exists; and 3) the dimensionality of the CoHOG feature vector is very large (e.g., 200,000). To deal with these problems, in this paper, we propose a framework that performs the following: 1) utilizes a novel gradient decomposition and combination strategy to make full use of the information of gradients; (2) adopts a two-stage gradient smoothing scheme to perform efficient gradient interpolation; and (3) employs incremental principal component analysis to reduce the large dimensionality of the CoHOG features. Experimental results on the two different human databases demonstrate the effectiveness of the proposed method.     

Modeling and rendering of realistic waterfall scenes with dynamic texture sprites

We propose an efficient approach for authoring dynamic and realistic waterfall scenes based on an acquired video sequence. Traditional video based techniques generate new images by synthesizing 2D samples, i.e., texture sprites chosen from a video sequence. However, they are limited to one fixed viewpoint and cannot provide arbitrary walkthrough into 3D scenes. Our approach extends this scheme by synthesizing dynamic 2D texture sprites and projecting them into 3D space. We first generate a set of basis texture sprites, which capture the representative appearance and motions of waterfall scenes contained in the video sequence. To model the shape and motion of a new waterfall scene, we interactively construct a set of flow lines taking account of physical principles. Along each flow line, the basis texture sprites are manipulated and animated dynamically, yielding a sequence of dynamic texture sprites in 3D space. These texture sprites are displayed using the point splatting technique, which can be accelerated efficiently by graphics hardware. By choosing varied basis texture sprites, waterfall scenes with different appearance and shapes can be conveniently simulated. The experimental results demonstrate that our approach achieves realistic effects and real‐time frame rates on consumer PC platforms. Copyright © 2006 John Wiley & Sons, Ltd.