Approximating Gradients for Meshes and Point Clouds via Diffusion Metric

The gradient of a function defined on a manifold is perhaps one of the most important differential objects in data analysis. Most often in practice, the input function is available only at discrete points sampled from the underlying manifold, and the manifold is approximated by either a mesh or simply a point cloud. While many methods exist for computing gradients of a function defined over a mesh, computing and simplifying gradients and related quantities such as critical points, of a function from a point cloud is non-trivial.
In this paper, we initiate the investigation of computing gradients under a different metric on the manifold from the original natural metric induced from the ambient space. Specifically, we map the input manifold to the eigenspace spanned by its Laplacian eigenfunctions, and consider the so-called diffusion distance metric associated with it. We show the relation of gradient under this metric with that under the original metric. It turns out that once the Laplace operator is constructed, it is easier to approximate gradients in the eigenspace for discrete inputs (especially point clouds) and it is robust to noises in the input function and in the underlying manifold. More importantly, we can easily smooth the gradient field at different scales within this eigenspace framework. We demonstrate the use of our new eigen-gradients with two applications: approximating / simplifying the critical points of a function, and the Jacobi sets of two input functions (which describe the correlation between these two functions), from point clouds data.     

流形距离与压缩感知核稀疏投影的局部线性嵌入算法

流形学习算法的目的是发现嵌入在高维数据空间中的低维表示,现有的流形学习算法对邻域参数k和噪声比较敏感。针对此问题,文中提出一种流形距离与压缩感知核稀疏投影的局部线性嵌入算法,其核心思想是集成局部线性嵌入算法对高维流形结构数据的降维有效性与压缩感知核稀疏投影的强鉴别性,以实现高效有降噪流形学习。首先,在选择各样本点的近邻域时,采用流形距离代替欧氏距离度量数据间相似度的方法,创建能够正确反映流形内部结构的邻域图,解决以欧氏距离作为相似性度量时对邻域参数的敏感。其次,利用压缩感知核稀疏投影作为从高维观测空间到低维嵌入空间的映射,增强算法的鉴别性。最后,利用Matlab工具对实验数据集进行仿真,进一步验证所提算法的有效性。     

Neural classification of finite elements

The work deals with a comparative performance of finite elements, making use of their formulation as vectors (or patterns) in a multi-dimensional space of proper attributes. Since the attributes control the performance, elements defined by similar patterns and related to the same class should show similar behavior. The pattern classification may be carried out with the help a self-organizing feature map of Kohonen with the patterns corresponding to the input space. These networks learn both the distribution and topology of a set of input space. At the end of the learning process, the neurons become selectively tuned to classes of input patterns, thus specifying “family relationships” among the elements. The work makes use of the four attributes: the element dimensionality, its number of nodes, maximum degree of interpolation polynomials and number of degrees of freedom per node, though a more general characterization is also possible.     

基于SOM网络的随机映射文本降维方法

文中针对在文本处理的高维矢量环境中Kohonen自组织特征映射神经网络的计算瓶颈问题进行分析,引入RM(随机映射)方法并进行相应的理论分析,在此基础上提出可以运用RM方法有效并且可控地解决上述计算瓶颈问题,降低了文本处理环境中Kohonen神经网络的规模和时间、空间代价。文章通过实验证明了上述方法的有效性和正确性,从而达到提高自组织理论对于文本处理的实时性和实际可行性的目的,并对其进一步应用进行展望。     

利用自组织特征映射神经网络进行可视化聚类

自组织特征映射作为一种神经网络方法,在数据挖掘、机器学习和模式分类中得到了广泛的应用。它将高维输人空间的数据映射到一个低维、规则的栅格上,从而可以利用可视化技术探测数据的固有特性。该文说明了自组织特征映射神经网络的工作原理和具体实现算法,同时利用一个算例展示了利用自组织特征映射进行聚类时的可视化特性,包括聚类过程的可视化和聚类结果的可视化,这也是自组织特征映射得到广泛应用的原因之一。     

Discrete Critical Values: a General Framework for Silhouettes Computation

Many shapes resulting from important geometric operations in industrial applications such as Minkowski sums or volume swept by a moving object can be seen as the projection of higher dimensional objects. When such a higher dimensional object is a smooth manifold, the boundary of the projected shape can be computed from the critical points of the projection. In this paper, using the notion of polyhedral chains introduced by Whitney, we introduce a new general framework to define an analogous of the set of critical points of piecewise linear maps defined over discrete objects that can be easily computed. We illustrate our results by showing how they can be used to compute Minkowski sums of polyhedra and volumes swept by moving polyhedra.     

Emotion recognition from geometric facial features using self-organizing map

This paper presents a novel emotion recognition model using the system identification approach. A comprehensive data driven model using an extended Kohonen self-organizing map (KSOM) has been developed whose input is a 26 dimensional facial geometric feature vector comprising eye, lip and eyebrow feature points. The analytical face model using this 26 dimensional geometric feature vector has been effectively used to describe the facial changes due to different expressions. This paper thus includes an automated generation scheme of this geometric facial feature vector. The proposed non-heuristic model has been developed using training data from MMI facial expression database. The emotion recognition accuracy of the proposed scheme has been compared with radial basis function network, multi-layered perceptron model and support vector machine based recognition schemes. The experimental results show that the proposed model is very efficient in recognizing six basic emotions while ensuring significant increase in average classification accuracy over radial basis function and multi-layered perceptron. It also shows that the average recognition rate of the proposed method is comparatively better than multi-class support vector machine.     

一种局部化的线性流形自组织映射

提出一种局部化的线性流形自组织映射方法, 可自主学习高维向量空间中的一组有序的低维线性流形. 与现有的基于Kohonen的自适应子空间自组织映射网络(Adaptive-subspace self-organizing map, ASSOM)方法相比较, 本文方法有效地克服了流形表达中出现的数据混淆现象, 网络中各神经元渐近学习各自区域内样本数据的平均向量和主元子空间, 数据表达更加清晰可辨. 实验中, 新方法对数据簇的分类准确率明显优于参与对比的其他三种方法, 其对手写体数字识别的准确率在MNIST训练集和测试集上分别达到了98.26\%和97.46\%.     

A Convolutional Deep Self-Organizing Map Feature extraction for machine learning

In this work we propose a new Unsupervised Deep Self-Organizing Map (UDSOM) algorithm for feature extraction, quite similar to the existing multi-layer SOM architectures. The principal underlying idea of using SOMs is that if a neuron is wins n times, these n inputs that activated this neuron are similar. The basic principle consists of an alternation of phases of splitting and abstraction of regions, based on a non-linear projection of high-dimensional data over a small space using Kohonen maps following a deep architecture. The proposed architecture consists of a splitting process, layers of alternating self-organizing, a rectification function RELU and an abstraction layer (convolution-pooling). The self-organizing layer is composed of a few SOMs with each map focusing on modelling a local sub-region. The most winning neurons of each SOM are then organized in a second sampling layer to generate a new 2D map. In parallel to this transmission of the winning neurons, an abstraction of the data space is obtained after the convolution-pooling module. The ReLU is then applied. This treatment is applied more than once, changing the size of the splitting window and the displacement step on the reconstructed input image each time. In this way, local information is gathered to form more global information in the upper layers by applying each time a convolution filter of the level. The architecture of the Unsupervised Deep Self-Organizing Map is unique and retains the same principle of deep learning algorithms. This architecture can be very interesting in a Big Data environment for machine learning tasks. Experiments have been conducted to discuss how the proposed architecture shows this performance.

     

Application of the Self-Organizing Feature Map Algorithm in Facial Image Morphing

A new facial image morphing algorithm based on the Kohonen self-organizing feature map (SOM) algorithm is proposed to generate a smooth 2D transformation that reflects anchor point correspondences. Using only a 2D face image and a small number of anchor points, we show that the proposed morphing algorithm provides a powerful mechanism for processing facial expressions.     

基于切空间判别学习的流形降维算法

在基于图像集的流形降维问题中,许多算法的核心思想都是把一个高维的流形直接降到一个维数相对较低、同时具有的判别信息更加充分的流形上.投影度量学习（projection metric learning,简称PML）是一种Grassmann流形降维算法.该算法是基于投影度量,并且使用RCG（Riemannian conjugate gradient）算法优化目标函数,其在多个数据集上都取得了较好的实验结果,但是对于复杂的人脸数据集,如YTC其实验结果相对较差,只取得了66.69%的正确率.同时,RCG算法的时间效率较差.基于上述原因,提出了基于切空间判别学习的流形降维算法.该算法首先对于PML中的投影矩阵添加扰动,使其成为对称正定（symmetric positive definite,简称SPD）矩阵;然后,使用LEM（log-euclidean metric）将其映射到切空间中;最后,利用基于特征值分解的迭代优化算法构造判别函数,得到变换矩阵.对提算法在多个标准数据集上进行了实验验证,并取得了较好的实验结果,从而验证了该算法的有效性.     

Growing a hypercubical output space in a self-organizing featuremap

Neural maps project data from an input space onto a neuron position in a (often lower dimensional) output space grid in a neighborhood preserving way, with neighboring neurons in the output space responding to neighboring data points in the input space. A map-learning algorithm can achieve an optimal neighborhood preservation only, if the output space topology roughly matches the effective structure of the data in the input space. We here present a growth algorithm, called the GSOM or growing self-organizing map, which enhances a widespread map self-organization process, Kohonen's self-organizing feature map (SOFM), by an adaptation of the output space grid during learning. The GSOM restricts the output space structure to the shape of a general hypercubical shape, with the overall dimensionality of the grid and its extensions along the different directions being subject of the adaptation. This constraint meets the demands of many larger information processing systems, of which the neural map can be a part. We apply our GSOM-algorithm to three examples, two of which involve real world data. Using recently developed methods for measuring the degree of neighborhood preservation in neural maps, we find the GSOM-algorithm to produce maps which preserve neighborhoods in a nearly optimal fashion.     

Smoothly distributed fuzzy c-means: a new self-organizing map

This paper presents a new self-organizing map algorithm. Unlike the well-known method of Kohonen, the new algorithm corresponds to the optimization of an unambiguously defined cost function. It consists of a modified version of the widely used fuzzy c-means functional, where the code vectors are distributed on a regular low-dimensional grid, and a penalization term is added in order to guarantee a smooth distribution for the values of the code vectors on the grid. The mapping properties of the new method, similar to those of Kohonen's algorithm, are illustrated with several data sets. Computer programs (source code and executables) and data are available upon request to the authors.     

A SOM-based hybrid linear-neural model for short-term load forecasting

In this paper, a short-term load forecasting method is considered, which is based upon a flexible smooth transition autoregressive (STAR) model. The described model is a linear model with time varying coefficients, which are the outputs of a single hidden layer feedforward neural network. The hidden layer is responsible for partitioning the input space into multiple sub-spaces through multivariate thresholds and smooth transition between the sub-spaces. In this paper, we propose a new method to smartly initialize the weights of the hidden layer of the neural network before its training. A self-organizing map (SOM) network is applied to split the historical data dynamics into clusters, and the Ho-Kashyap algorithm is then used to obtain the separating planes' equations. Applied to the electricity markets, the proposed method is better able to model the smooth transitions between the different regimes, which are present in the load demand series because of market effects and season effects. We use data from three electricity markets to compare the prediction accuracy of the proposed method with traditional benchmarks and other recent models, and find our results to be competitive.     

Adaptive nonlinear manifolds and their applications to pattern recognition

Dimensionality reduction has long been associated with retinotopic mapping for understanding cortical maps. Multisensory information is processed, fused and mapped to an essentially 2-D cortex in an information preserving manner. Data processing and projection techniques inspired by this biological mechanism are playing an increasingly important role in pattern recognition, computational intelligence, data mining, information retrieval and image recognition. Dimensionality reduction involves reduction of features or volume of data and has become an essential step of information processing in many fields. The topic of manifold learning has recently attracted a great deal of attention, and a number of advanced techniques for extracting nonlinear manifolds and reducing data dimensions have been proposed from statistics, geometry theory and adaptive neural networks. This paper provides an overview of this challenging and emerging topic and discusses various recent methods such as self-organizing map (SOM), kernel PCA, principal manifold, isomap, local linear embedding, and Laplacian eigenmap. Many of them can be considered in a learning manifold framework. The paper further elaborates on the biologically inspired SOM model and its metric preserving variant ViSOM under the framework of adaptive manifold; and their applications in dimensionality reduction with face recognition are investigated. The experiments demonstrate that adaptive ViSOM-based methods produce markedly improved performance over the others due to their metric scaling and preserving properties along the nonlinear manifold.     

A self-organizing map with twin units capable of describing a nonlinear input-output relation applied to speech code vector mapping

This paper describes a new type of self-organizing map (SOM) with twin units as opposed to the single unit type conventional SOM proposed by Kohonen. The present self-organizing map with twin units (TW-SOM) can describe a nonlinear input-output relation with high accuracy. It is applied to voice conversion problem from bone conduction voice to air conduction voice (nonlinear code vector mapping), and its superiority over the conventional method using Linde-Buzo-Gray (LBG) algorithm is discussed. The tone quality of the converted voice is examined not only from the quantization distortion viewpoint, but also from the auditory sensation viewpoint through actual listening tests. The enhancement of the tone quality was experimentally confirmed.     

A new model of self-organizing neural networks and its applicationin data projection

In this paper a new model of self-organizing neural networks is proposed. An algorithm called "double self-organizing feature map" (DSOM) algorithm is developed to train the novel model. By the DSOM algorithm the network will adaptively adjust its network structure during the learning phase so as to make neurons responding to similar stimulus have similar weight vectors and spatially move nearer to each other at the same time. The final network structure allows us to visualize high-dimensional data as a two dimensional scatter plot. The resulting representations allow a straightforward analysis of the inherent structure of clusters within the input data. One high-dimensional data set is used to test the effectiveness of the proposed neural networks.     

Visualization of Tree-Structured Data Through Generative Topographic Mapping

In this paper, we present a probabilistic generative approach for constructing topographic maps of tree-structured data. Our model defines a low-dimensional manifold of local noise models, namely, (hidden) Markov tree models, induced by a smooth mapping from low-dimensional latent space. We contrast our approach with that of topographic map formation using recursive neural-based techniques, namely, the self-organizing map for structured data (SOMSD) (Hagenbuchner , 2003). The probabilistic nature of our model brings a number of benefits: 1) naturally defined cost function that drives the model optimization; 2) principled model comparison and testing for overfitting; 3) a potential for transparent interpretation of the map by inspecting the underlying local noise models; 4) natural accommodation of alternative local noise models implicitly expressing different notions of structured data similarity. Furthermore, in contrast with the recursive neural-based approaches, the smooth nature of the mapping from the latent space to the local model space allows for calculation of magnification factors—a useful tool for the detection of data clusters. We demonstrate our approach on three data sets: a toy data set, an artificially generated data set, and on a data set of images represented as quadtrees.