Shape transformer nets: Generating viewpoint-invariant 3D shapes from a single image

Single-view 3D shapes generation has achieved great success in recent years. However, current methods always blind the learning of shapes and viewpoints. The generated shape only fit the observed viewpoints and would not be optimal from unknown viewpoints. In this paper, we propose a novel encoder–decoder based network which contains a disentangled transformer to generate the viewpoint-invariant 3D shapes. The differentiable and parametric Non-uniform B-spline (NURBS) surface generation and 3D-to-3D viewpoint transformation are incorporated to learn the viewpoint-invariant shape and the camera viewpoint, respectively. Our new framework allows us to learn the latent geometric parameters of shapes and viewpoints without knowing the ground truth viewpoint. That can simultaneously generate camera-viewpoint and viewpoint-invariant 3D shapes of the object. We analyze the effects of disentanglement and show both quantitative and qualitative results of shapes generated at various unknown viewpoints.     

基于形状匹配变形模型的三维人脸重构

为了从多幅人脸图像构造三维人脸结构,通常需要自动提取不同图像中的对应特征点,这往往是很难完成的.为了避免这个困难,本文建立了一个基于形状匹配的三维变形模型,在保证形状最佳匹配的条件下,实现对人脸图像姿态的估计和三维人脸重构.模型采用径向基函数对通用头部模型进行变形,用形状上下文来描述点之间的形状相似性,形状距离用来描述头部模型和人脸图像整体形状上的相似性,从而实现形状最佳匹配意义上的三维重构.实验表明,本文的算法只需要在人脸图像中提取特征点集,不需进行配准,就可以恢复出令人满意的三维头部结构.     

Programmed Shape‐Morphing Scaffolds Enabling Facile 3D Endothelialization

Rapid formation of a confluent endothelial monolayer is the key to the success of small‐diameter vascular grafts, which is significantly important for treating dangerous and even sometimes deadly vascular disorders. However, the difficulty to homogenously locate endothelial cells onto the lumen of small‐diameter tubular scaffolds makes 3D endothelialization greatly challenging. Here, novel shape‐morphing scaffolds enabling programmed deformation from planar shapes to small‐diameter tubular shapes are designed and developed by combining biocompatible shape memory polymer and electrospun nanofibrous membrane. Endothelial cells can be conveniently seeded and attached on the 2D surface of the scaffolds and subsequently self‐rolled into 3D organization at physiological temperature. Endothelial cell responses and functions are varied on the shape‐morphing scaffolds with different nanofibrous electrospun membranes as the inner layer, arisen from the inducement of scaffolds with different morphological, physical, and biochemical characteristics. Owing to excellent properties of the nanofibrous membrane fabricated by the coelectrospinning of poly‐ε‐caprolactone (PCL) and gelatin methacrylate (GelMA), the shape‐morphing scaffolds with a nanofibrous PCL/GelMA inner layer support desirable homogeneous endothelial cell attachment as well as the rapid formation of biomimetic cell–scaffold interaction and cell–cell interaction under the 3D cell culture condition, therefore offering a visible approach for facile 3D endothelialization.     

Shape-Memory Balloon Structures by Pneumatic Multi-material 4D Printing

4D printing is an attractive approach for manufacturing structures that can adopt new shapes or functionalities after printing. However, 4D printing methods and materials that can be used to achieve structures with complex shapes and excellent mechanical properties simultaneously are still lacking. Here, a novel 4D printing is developed where multi-material digital light process 3D printing of shape memory polymers (SMPs) fabricates a structure that is later transformed into a complex 3D shape with robust mechanical properties by pneumatic manipulation. In this method, the shape change is controlled by the spatial distributions of SMPs, which is designed by finite element analysis. Experimental investigations are carried out to print various structured balloons with predefined intricate shapes, including a structure in dog-like shape and a surface with the human face contour. These structures are also endowed with robust mechanical stiffness and lightweight features, which allow this new 4D printing approach for potential applications in biomedical devices, reconfigurable structures, and metamaterials.     

基于球谐分析和三维矩的超二次模型匹配计算

超二次模型能够利用简洁的参数有效地描述大量复杂的、真实的三维形状,因此,在计算机视觉和计算机图形中都得到了广泛的应用。超二次模型形状匹配计算是应用超二次模型进行三维物体识别的一个关键问题。本文对此进行了详细的研究,提出了两种新的方法,即基于球谐分析和三维矩的匹配计算方法。特别地,在基于球谐分析的方法中,提出了一个新的采样算法,用以构造描述超二次模型的离散球函数。实验表明,本文提出的方法能够有效地进行超二次模型三维形状的匹配计算,为开发基于超二次模型的三维物体识别系统奠定了良好的基础。     

Colorful 3D reconstruction at high resolution using multi-view representation

High-quality 3D models should contain accurate shapes, as well as other correct attributes, such as realistic surface color. However, current researches were mostly focused on the reconstruction of shapes. We present a method to reconstruct high-resolution colorful 3D models from single images. Shapes and colors are learned separately, using a coarse-to-fine strategy in which the 3D color is expressed as 3-channel volumes. Colorful volumes share the same spatial dimension with generated shape volumes. We propose orthographic colorful maps to retain and recover projected coordinates and corresponding color for 3D surface points. To achieve a fine granularity increase in the quality of maps from low-resolution to high-resolution, we introduce 2D super resolution during reconstructing 3D shapes and color volumes. Models are carved by utilizing predicted high-resolution silhouette, depth and color details. Experimental results in a subset of the ShapeNet dataset and the Colorful Human dataset show the effectiveness of our method.     

Collaborative Distribution Alignment for 2D image-based 3D shape retrieval

Retrieving 3D shapes with 2D images has become a popular research area nowadays, and a great deal of work has been devoted to reducing the discrepancy between 3D shapes and 2D images to improve retrieval performance. However, most approaches ignore the semantic information and decision boundaries of the two domains, and cannot achieve both domain alignment and category alignment in one module. In this paper, a novel Collaborative Distribution Alignment (CDA) model is developed to address the above existing challenges. Specifically, we first adopt a dual-stream CNN, following a similarity guided constraint module, to generate discriminative embeddings for input 2D images and 3D shapes (described as multiple views). Subsequently, we explicitly introduce a joint domain-class alignment module to dynamically learn a class-discriminative and domain-agnostic feature space, which can narrow the distance between 2D image and 3D shape instances of the same underlying category, while pushing apart the instances from different categories. Furthermore, we apply a decision boundary refinement module to avoid generating class-ambiguity embeddings by dynamically adjusting inconsistencies between two discriminators. Extensive experiments and evaluations on two challenging benchmarks, MI3DOR and MI3DOR-2, demonstrate the superiority of the proposed CDA method for 2D image-based 3D shape retrieval task.     

利用网格卷积特征的三维形变目标分类

三维目标的形状变化给目标识别带来很大挑战,同时三维网格模型的不规则数据结构难以直接应用卷积运算提取三维目标特征.对此,本文提出了一种高效的三维形变目标的网格卷积特征表示方法,准确提取形状信息并进行分类.首先通过网格卷积运算获得形变目标中典型局部曲面形状分布,其次通过马尔科夫链对曲面形状的空间共现关系建模,从而形成三维模型的全局特征描述,最后采用支持向量机实现形变目标分类.该方法将连续多项式函数作为卷积模板,实现针对不规则数据结构的网格卷积运算,并且给出了卷积模板参数的无监督学习方法.在标准非刚性三维模型数据集SHREC10与SHREC15上的实验结果表明本文方法能有效提取三维网格模型的形状信息,分类准确率分别达到了92.88%与96.54%.     

T-S型RBF神经网络在红外火焰探测系统中的应用

基于T-S(Takagi-Sugeno,高木-关野)模型RBF(Radial Basis Function,径向基函数)神经网络,提出了一种应用于三波段点型红外火焰探测器的识别算法,同时实现了硬件电路以及软件程序的设计。针对火焰探测器在检测过程中可能出现的数据丢失、失真、饱和等复杂情况,本文利用RBF网络较优的逼近精度和泛化能力,同时结合T-S模型用少量的模糊规则可生成较复杂的非线性函数的特点,实现了火焰与干扰源的准确识别。实验证实,T-S模型RBF神经网络相比于BP(Back Propagation,反向传播)网络在逼近精度、收敛速度、鲁棒性等多个方面都有所提升。     

二氧化硅光波导刻蚀参量及掩膜的优化

硅基二氧化硅光波导是光通信中的关键器件.采用光刻胶以及金属作为掩膜进行了反应离子刻蚀二氧化硅光波导的工艺研究,获得了刻蚀速率及刻蚀选择比相对各工艺参数变化的三维神经网络模型.利用一种新型的用于二氧化硅深刻蚀的复合双层掩膜结构,克服了许多单层掩膜自身的限制,并利用这一结构制作出低传输损耗的硅基二氧化硅波导.     

基于3D骨架和RGB图片的时空动作识别

基于3D骨架的动作识别技术现已成为人机交互的重要手段。为了提高3D动作识别的精度,文中提出一种将3D骨架特征和2D图片特征进行融合的双流神经网络。其中一个网络处理3D骨架序列,另一个网络处理2D图片。最后再将二者的特征进行融合,以提高识别精度。相较于单独使用3D骨架的动作识别,文中所使用的方法在NTU_RGBD数据集以及SYSU数据集上都有了很大的精度提升。     

三维物体形状检测与重构系统的研究

三维物体形状检测与重构技术是计算机图像处理技术的一个分支,在众多领域有着广泛的应用前景。文中介绍了基于面结构光投影法的三维物体形状检测与重构系统,阐述了测量原理,建立了数学模型,给出了系统检测与重构的步骤,并根据检测得到的三维点云数据的特点,提出了适合本课题的简单、有效的三角网格化方法。实验结果证明该系统能够进行有效的检测与重构,具有很好的应用价值。     

Environment-adaptation mobile radio propagation prediction usingradial basis function neural networks

This paper investigates the application of a radial basis function (RBF) neural network to the prediction of field strength based on topographical and morphographical data. The RBF neural network is a two-layer localized receptive field network whose output nodes from a combination of radial activation functions computed by the hidden layer nodes. Appropriate centers and connection weights in the RBF network lead to a network that is capable of forming the best approximation to any continuous nonlinear mapping up to an arbitrary resolution. Such an approximation introduces best nonlinear approximation capability into the prediction model in order to accurately predict propagation loss over an arbitrary environment based on adaptive learning from measurement data. The adaptive learning employs hybrid competitive and recursive least squares algorithms. The unsupervised competitive algorithm adjusts the centers while the recursive least squares (RLS) algorithm estimates the connection weights. Because these two learning rules are both linear, rapid convergence is guaranteed. This hybrid algorithm significantly enhances the real-time or adaptive capability of the RBF-based prediction model. The applications to Okumura's (1968) data are included to demonstrate the effectiveness of the RBF neural network approach     

基于视图感知的单视图三维重建算法

尽管由于丢弃维度将3维(3D)形状投影到2维(2D)视图看似是不可逆的,但是从可视化到计算机辅助几何设计,各个垂直行业对3维重建技术的兴趣正迅速增长。传统基于物体深度图或者RGB图的3维重建算法虽然可以在一些方面达到令人满意的效果,但是它们仍然面临若干问题:(1)粗鲁的学习2D视图与3D形状之间的映射;(2)无法解决物体不同视角下外观差异所带来的的影响;(3)要求物体多个观察视角下的图像。该文提出一个端到端的视图感知3维(VA3D)重建网络解决了上述问题。具体而言,VA3D包含多邻近视图合成子网络和3D重建子网络。多邻近视图合成子网络基于物体源视图生成多个邻近视角图像,且引入自适应融合模块解决了视角转换过程中出现的模糊或扭曲等问题。3D重建子网络使用循环神经网络从合成的多视图序列中恢复物体3D形状。通过在ShapeNet数据集上大量定性和定量的实验表明,VA3D有效提升了基于单视图的3维重建结果。     

Combining 3D Printing with Electrospinning for Rapid Response and Enhanced Designability of Hydrogel Actuators

Porous structures have emerged as a breakthrough of shape‐morphing hydrogels to achieve a rapid response. However, these porous actuators generally suffer from a lack of complexity and diversity in obtained 3D shapes. Herein, a simple yet versatile strategy is developed to generate shape‐morphing hydrogels with both fast deformation and enhanced designability in 3D shapes by combining two promising technologies: electrospinning and 3D printing. Elaborate patterns are printed on mesostructured stimuli‐responsive electrospun membranes, modulating in‐plane and interlayer internal stresses induced by swelling/shrinkage mismatch, and thus guiding morphing behaviors of electrospun membranes to adapt to changes of the environment. With this strategy, a series of fast deformed hydrogel actuators are constructed with various distinctive responsive behaviors, including reversible/irreversible formations of 3D structures, folding of 3D tubes, and formations of 3D structures with multi low‐energy states. It is worth noting that although poly(N‐isopropyl acrylamide) is chosen as the model system in the present research, our strategy is applicable to other stimuli‐responsive hydrogels, which enriches designs of rapid deformed hydrogel actuators.     

基于RBFNN和GA优化设计DGS微带*线

采用神经网络与遗传算法相结合的方法,对一种缺陷接地结构微带线进行优化设计。整个优化设计过程分为两步进行：首先采用基于混合递阶遗传算法优化训练的RBF神经网络对缺陷接地结构微带线进行建模,当神经网络模型训练成功后就实现了对这种缺陷地结构微带线传输系数快速和精确的输出仿真;然后利用该神经网络模型,并结合遗传算法共同优化设计这种缺陷地结构微带线的传输系数和尺寸参数。仿真结果验证了该方法的有效性和准确性。     

Image compression optimized for 3D reconstruction by utilizing deep neural networks

Computer vision tasks are often expected to be executed on compressed images. Classical image compression standards like JPEG 2000 are widely used. However, they do not account for the specific end-task at hand. Motivated by works on recurrent neural network (RNN)-based image compression and three-dimensional (3D) reconstruction, we propose unified network architectures to solve both tasks jointly. These joint models provide image compression tailored for the specific task of 3D reconstruction. Images compressed by our proposed models, yield 3D reconstruction performance superior as compared to using JPEG 2000 compression. Our models significantly extend the range of compression rates for which 3D reconstruction is possible. We also show that this can be done highly efficiently at almost no additional cost to obtain compression on top of the computation already required for performing the 3D reconstruction task.