A Survey on Shape Correspondence

We review methods designed to compute correspondences between geometric shapes represented by triangle meshes, contours or point sets. This survey is motivated in part by recent developments in space–time registration, where one seeks a correspondence between non‐rigid and time‐varying surfaces, and semantic shape analysis, which underlines a recent trend to incorporate shape understanding into the analysis pipeline. Establishing a meaningful correspondence between shapes is often difficult because it generally requires an understanding of the structure of the shapes at both the local and global levels, and sometimes the functionality of the shape parts as well. Despite its inherent complexity, shape correspondence is a recurrent problem and an essential component of numerous geometry processing applications. In this survey, we discuss the different forms of the correspondence problem and review the main solution methods, aided by several classification criteria arising from the problem definition. The main categories of classification are defined in terms of the input and output representation, objective function and solution approach. We conclude the survey by discussing open problems and future perspectives.     

基于深度学习的三维数据分析理解方法研究综述

基于深度学习的三维数据分析理解是数字几何领域的一个研究热点.不同于基于深度学习的图像分析理解,基于深度学习的三维数据分析理解需要解决的首要问题是数据表达的多样性.相较于规则的二维图像,三维数据有离散表达和连续表达的方法,目前基于深度学习的相关工作多基于三维数据的离散表示,不同的三维数据表达方法与不同的数字几何处理任务对深度学习网络的要求也不同.本文首先汇总了常用的三维数据集与特定任务的评价指标,并分析了三维模型特征描述符.然后从特定任务出发,就不同的三维数据表达方式,对现有的基于深度学习的三维数据分析理解网络进行综述,对各类方法进行对比分析,并从三维数据表达方法的角度进一步汇总现有工作.最后基于国内外研究现状,讨论了亟待解决的挑战性问题,展望了未来发展的趋势.     

Freeform Shape Representations for Efficient Geometry Processing

The most important concepts for the handling and storage of freeform shapes in geometry processing applications are parametric representations and volumetric representations. Both have their specific advantages and drawbacks. While the algebraic complexity of volumetric representations is independent from the shape complexity, the domain of a parametric representation usually has to have the same structure as the surface itself (which sometimes makes it necessary to update the domain when the surface is modified). On the other hand, the topology of a parametrically defined surface can be controlled explicitly while in a volumetric representation, the surface topology can change accidentally during deformation. A volumetric representation reduces distance queries or inside/outside tests to mere function evaluations but the geodesic neighborhood relation between surface points is difficult to resolve. As a consequence, it seems promising to combine parametric and volumetric representations to effectively exploit both advantages. In this talk, a number of projects are presented and discussed in which such a combination leads to efficient and numerically stable algorithms for the solution of various geometry processing tasks. Applications include global error control for mesh decimation and smoothing, topology control for level‐set surfaces, and shape modeling with unstructured point clouds.     

Cross-media similarity metric learning with unified deep networks

As a highlighting research topic in the multimedia area, cross-media retrieval aims to capture the complex correlations among multiple media types. Learning better shared representation and distance metric for multimedia data is important to boost the cross-media retrieval. Motivated by the strong ability of deep neural network in feature representation and comparison functions learning, we propose the Unified Network for Cross-media Similarity Metric (UNCSM) to associate cross-media shared representation learning with distance metric in a unified framework. First, we design a two-pathway deep network pretrained with contrastive loss, and employ double triplet similarity loss for fine-tuning to learn the shared representation for each media type by modeling the relative semantic similarity. Second, the metric network is designed for effectively calculating the cross-media similarity of the shared representation, by modeling the pairwise similar and dissimilar constraints. Compared to the existing methods which mostly ignore the dissimilar constraints and only use sample distance metric as Euclidean distance separately, our UNCSM approach unifies the representation learning and distance metric to preserve the relative similarity as well as embrace more complex similarity functions for further improving the cross-media retrieval accuracy. The experimental results show that our UNCSM approach outperforms 8 state-of-the-art methods on 4 widely-used cross-media datasets.

     

Differential Representations for Mesh Processing

Surface representation and processing is one of the key topics in computer graphics and geometric modeling, since it greatly affects the range of possible applications. In this paper we will present recent advances in geometry processing that are related to the Laplacian processing framework and differential representations. This framework is based on linear operators defined on polygonal meshes, and furnishes a variety of processing applications, such as shape approximation and compact representation, mesh editing, watermarking and morphing. The core of the framework is the definition of differential coordinates and new bases for efficient mesh geometry representation, based on the mesh Laplacian operator.     

String‐Based Synthesis of Structured Shapes

We propose a novel method to synthesize geometric models from a given class of context‐aware structured shapes such as buildings and other man‐made objects. The central idea is to leverage powerful machine learning methods from the area of natural language processing for this task. To this end, we propose a technique that maps shapes to strings and vice versa, through an intermediate shape graph representation. We then convert procedurally generated shape repositories into text databases that, in turn, can be used to train a variational autoencoder. The autoencoder enables higher level shape manipulation and synthesis like, for example, interpolation and sampling via its continuous latent space. We provide project code and pre‐trained models.     

Deep‐learning the Latent Space of Light Transport

We suggest a method to directly deep‐learn light transport, i. e., the mapping from a 3D geometry‐illumination‐material configuration to a shaded 2D image. While many previous learning methods have employed 2D convolutional neural networks applied to images, we show for the first time that light transport can be learned directly in 3D. The benefit of 3D over 2D is, that the former can also correctly capture illumination effects related to occluded and/or semi‐transparent geometry. To learn 3D light transport, we represent the 3D scene as an unstructured 3D point cloud, which is later, during rendering, projected to the 2D output image. Thus, we suggest a two‐stage operator comprising a 3D network that first transforms the point cloud into a latent representation, which is later on projected to the 2D output image using a dedicated 3D‐2D network in a second step. We will show that our approach results in improved quality in terms of temporal coherence while retaining most of the computational efficiency of common 2D methods. As a consequence, the proposed two stage‐operator serves as a valuable extension to modern deferred shading approaches.     

局部几何与全局结构联合感知的三维形状分类方法

针对复杂结构的三维形状分析与识别问题,提出了新颖的图卷积分类方法,建立了局部几何与全局结构联合图卷积学习机制,有效提高了三维形状数据学习的鲁棒性与稳定性。首先,通过最远点采样与最近邻方法构造局部图,并建立动态卷积算子,有效提取局部几何特征;同时,基于特征域采样构造全局的特征谱图,通过卷积算子获得全局结构信息。进而,构建加权的联合图卷积学习网络模型,引入注意力机制,实现自适应的特征融合。最终,在联合优化目标函数约束下,有效提高特征学习的性能。实验结果表明,融合局部几何与全局结构的联合图卷积网络学习机制,有效提高了深度特征的表示能力及区分性,具有更为优秀的识别力和分类性能。提出的研究方法可应用于大规模三维场景识别、三维重建以及数据压缩,在机器人、产品数字化分析、智能导航、虚拟现实等领域具有着重要的工程意义与广泛的应用前景。     

自监督深度残差函数映射网络的3维模型对应关系计算

目的 针对传统非刚性3维模型的对应关系计算方法需要模型间真实对应关系监督的缺点,提出一种自监督深度残差函数映射网络（self-supervised deep residual functional maps network,SSDRFMN）。方法 首先将局部坐标系与直方图结合以计算3维模型的特征描述符,即方向直方图签名（signature of histograms of orientations,SHOT）描述符;其次将源模型与目标模型的SHOT描述符输入SSDRFMN,利用深度函数映射（deep functional maps,DFM）层计算两个模型间的函数映射矩阵,并通过模糊对应层将函数映射关系转换为点到点的对应关系;最后利用自监督损失函数计算模型间的测地距离误差,对计算出的对应关系进行评估。结果 实验结果表明,在MPI-FAUST数据集上,本文算法相比于有监督的深度函数映射（supervised deep functional maps,SDFM）算法,人体模型对应关系的测地误差减小了1.45;相比于频谱上采样（spectral upsampling,SU）算法减小了1.67。在TOSCA数据集上,本文算法相比于SDFM算法,狗、猫和狼等模型的对应关系的测地误差分别减小了3.13、0.98和1.89;相比于SU算法分别减小了2.81、2.22和1.11,并有效克服了已有深度函数映射方法需要模型间的真实对应关系来监督的缺点,使得该方法可以适用于不同的数据集,可扩展性大幅增强。结论 本文通过自监督深度残差函数映射网络训练模型的方向直方图签名描述符,提升了模型对应关系的准确率。本文方法可以适应于不同的数据集,相比传统方法,普适性较好。     

PointProNets: Consolidation of Point Clouds with Convolutional Neural Networks

With the widespread use of 3D acquisition devices, there is an increasing need of consolidating captured noisy and sparse point cloud data for accurate representation of the underlying structures. There are numerous algorithms that rely on a variety of assumptions such as local smoothness to tackle this ill‐posed problem. However, such priors lead to loss of important features and geometric detail. Instead, we propose a novel data‐driven approach for point cloud consolidation via a convolutional neural network based technique. Our method takes a sparse and noisy point cloud as input, and produces a dense point cloud accurately representing the underlying surface by resolving ambiguities in geometry. The resulting point set can then be used to reconstruct accurate manifold surfaces and estimate surface properties. To achieve this, we propose a generative neural network architecture that can input and output point clouds, unlocking a powerful set of tools from the deep learning literature. We use this architecture to apply convolutional neural networks to local patches of geometry for high quality and efficient point cloud consolidation. This results in significantly more accurate surfaces, as we illustrate with a diversity of examples and comparisons to the state‐of‐the‐art.     

Image‐to‐Geometry Registration: a Mutual Information Method exploiting Illumination‐related Geometric Properties

This work concerns a novel study in the field of image‐to‐geometry registration. Our approach takes inspiration from medical imaging, in particular from multi‐modal image registration. Most of the algorithms developed in this domain, where the images to register come from different sensors (CT, X‐ray, PET), are based on Mutual Information, a statistical measure of non‐linear correlation between two data sources. The main idea is to use mutual information as a similarity measure between the image to be registered and renderings of the model geometry, in order to drive the registration in an iterative optimization framework. We demonstrate that some illumination‐related geometric properties, such as surface normals, ambient occlusion and reflection directions can be used for this purpose. After a comprehensive analysis of such properties we propose a way to combine these sources of information in order to improve the performance of our automatic registration algorithm. The proposed approach can robustly cover a wide range of real cases and can be easily extended.     

Limit Shapes – A Tool for Understanding Shape Differences and Variability in 3D Model Collections

We propose a novel construction for extracting a central or limit shape in a shape collection, connected via a functional map network. Our approach is based on enriching the latent space induced by a functional map network with an additional natural metric structure. We call this shape‐like dual object the limit shape and show that its construction avoids many of the biases introduced by selecting a fixed base shape or template. We also show that shape differences between real shapes and the limit shape can be computed and characterize the unique properties of each shape in a collection – leading to a compact and rich shape representation. We demonstrate the utility of this representation in a range of shape analysis tasks, including improving functional maps in difficult situations through the mediation of limit shapes, understanding and visualizing the variability within and across different shape classes, and several others. In this way, our analysis sheds light on the missing geometric structure in previously used latent functional spaces, demonstrates how these can be addressed and finally enables a compact and meaningful shape representation useful in a variety of practical applications.     

Shape‐Up: Shaping Discrete Geometry with Projections

We introduce a unified optimization framework for geometry processing based on shape constraints. These constraints preserve or prescribe the shape of subsets of the points of a geometric data set, such as polygons, one‐ring cells, volume elements, or feature curves. Our method is based on two key concepts: a shape proximity function and shape projection operators. The proximity function encodes the distance of a desired least‐squares fitted elementary target shape to the corresponding vertices of the 3D model. Projection operators are employed to minimize the proximity function by relocating vertices in a minimal way to match the imposed shape constraints. We demonstrate that this approach leads to a simple, robust, and efficient algorithm that allows implementing a variety of geometry processing applications, simply by combining suitable projection operators. We show examples for computing planar and circular meshes, shape space exploration, mesh quality improvement, shape‐preserving deformation, and conformal parametrization. Our optimization framework provides a systematic way of building new solvers for geometry processing and produces similar or better results than state‐of‐the‐art methods.     

Generalized sparse metric learning with relative comparisons

The objective of sparse metric learning is to learn a distance measure from a set of data in addition to finding a low-dimensional representation. Despite demonstrated success, the performance of existing sparse metric learning approaches is usually limited because the methods assumes certain problem relaxations or they target the SML objective indirectly. In this paper, we propose a Generalized Sparse Metric Learning method. This novel framework offers a unified view for understanding many existing sparse metric learning algorithms including the Sparse Metric Learning framework proposed in (Rosales and Fung ACM International conference on knowledge discovery and data mining (KDD), pp 367–373, 2006), the Large Margin Nearest Neighbor (Weinberger et al. in Advances in neural information processing systems (NIPS), 2006; Weinberger and Saul in Proceedings of the twenty-fifth international conference on machine learning (ICML-2008), 2008), and the D-ranking Vector Machine (D-ranking VM) (Ouyang and Gray in Proceedings of the twenty-fifth international conference on machine learning (ICML-2008), 2008). Moreover, GSML also establishes a close relationship with the Pairwise Support Vector Machine (Vert et al. in BMC Bioinform, 8, 2007). Furthermore, the proposed framework is capable of extending many current non-sparse metric learning models to their sparse versions including Relevant Component Analysis (Bar-Hillel et al. in J Mach Learn Res, 6:937–965, 2005) and a state-of-the-art method proposed in (Xing et al. Advances in neural information processing systems (NIPS), 2002). We present the detailed framework, provide theoretical justifications, build various connections with other models, and propose an iterative optimization method, making the framework both theoretically important and practically scalable for medium or large datasets. Experimental results show that this generalized framework outperforms six state-of-the-art methods with higher accuracy and significantly smaller dimensionality for seven publicly available datasets.     

基于RefineNet的端到端语音增强方法

为提高神经网络对语音信号时域波形的直接处理能力,提出了一种基于RefineNet的端到端语音增强方法.本文构建了一个时频分析神经网络,模拟语音信号处理中的短时傅里叶变换,利用RefineNet网络学习含噪语音到纯净语音的特征映射.在模型训练阶段,用多目标联合优化的训练策略将语音增强的评价指标短时客观可懂度(Short-time objective intelligibility,STOI)与信源失真比(Source to distortion ratio,SDR)融入到训练的损失函数.在与具有代表性的传统方法和端到端的深度学习方法的对比实验中,本文提出的算法在客观评价指标上均取得了最好的增强效果,并且在未知噪声和低信噪比条件下表现出更好的抗噪性.     

基于RefineNet的端到端语音增强方法

为提高神经网络对语音信号时域波形的直接处理能力,提出了一种基于RefineNet的端到端语音增强方法.本文构建了一个时频分析神经网络,模拟语音信号处理中的短时傅里叶变换,利用RefineNet网络学习含噪语音到纯净语音的特征映射.在模型训练阶段,用多目标联合优化的训练策略将语音增强的评价指标短时客观可懂度(Short-time objective intelligibility,STOI)与信源失真比(Source to distortion ratio,SDR)融入到训练的损失函数.在与具有代表性的传统方法和端到端的深度学习方法的对比实验中,本文提出的算法在客观评价指标上均取得了最好的增强效果,并且在未知噪声和低信噪比条件下表现出更好的抗噪性.     

A new approach to applicable memory-based reasoning

To appropriately utilize the rapidly growing amount of data and information is a big challenge for people and organizations. Standard information retrieval methods, using sequential processing combined with syntax-based indexing and access methods, have not been able to adequately handle this problem. We are currently investigating a different approach, based on a combination of massive parallel processing with case-based (memory-based) reasoning methods. Given the problems of purely syntax-based retrieval methods, we suggest ways of incorporating general domain knowledge into memory-based reasoning. Our approach is related to the properties of the parallel processing microchip MS160, particularly targeted at fast information retrieval from very large data sets. Within this framework different memory-based methods are studied, differing in the type and representation of cases, and in the way that the retrieval methods are supported by explicit general domain knowledge. Cases can be explicitly stored information retrieval episodes, virtually stored abstractions linked to document records, or merely the document records themselves. General domain knowledge can be a multi-relational semantic network, a set of term dependencies and relevances, or compiled into a global similarity metric. This paper presents the general framework, discusses the core issues involved, and describes three different methods illustrated by examples from the domain of medical diagnosis.     

A systematic review on deep learning architectures and applications

The amount of digital data in the universe is growing at an exponential rate, doubling every 2 years, and changing how we live in the world. The information storage capacity and data requirement crossed the zettabytes. With this level of bombardment of data on machine learning techniques, it becomes very difficult to carry out parallel computations. Deep learning is broadening its scope and gaining more popularity in natural language processing, feature extraction and visualization, and almost in every machine learning trend. The purpose of this study is to provide a brief review of deep learning architectures and their working. Research papers and proceedings of conferences from various authentic resources (Institute of Electrical and Electronics Engineers, Wiley, Nature, and Elsevier) are studied and analyzed. Different architectures and their effectiveness to solve domain specific problems are evaluated. Various limitations and open problems of current architectures are discussed to provide better insights to help researchers and student to resume their research on these issues. One hundred one articles were reviewed for this meta‐analysis of deep learning. From this analysis, it is concluded that advanced deep learning architectures are combinations of few conventional architectures. For example, deep belief network and convolutional neural network are used to build convolutional deep belief network, which has higher capabilities than the parent architectures. These combined architectures are more robust to explore the problem space and thus can be the answer to build a general‐purpose architecture.