3D model reconstruction using neural gas accelerated on GPU

In this work, we propose the use of the neural gas (NG), a neural network that uses an unsupervised Competitive Hebbian Learning (CHL) rule, to develop a reverse engineering process. This is a simple and accurate method to reconstruct objects from point clouds obtained from multiple overlapping views using low-cost sensors. In contrast to other methods that may need several stages that include downsampling, noise filtering and many other tasks, the NG automatically obtains the 3D model of the scanned objects. To demonstrate the validity of our proposal we tested our method with several models and performed a study of the neural network parameterization computing the quality of representation and also comparing results with other neural methods like growing neural gas and Kohonen maps or classical methods like Voxel Grid. We also reconstructed models acquired by low cost sensors that can be used in virtual and augmented reality environments for redesign or manipulation purposes. Since the NG algorithm has a strong computational cost we propose its acceleration. We have redesigned and implemented the NG learning algorithm to fit it onto Graphics Processing Units using CUDA. A speed-up of 180× faster is obtained compared to the sequential CPU version.     

GPGPU implementation of growing neural gas: Application to 3D scene reconstruction

Self-organising neural models have the ability to provide a good representation of the input space. In particular the Growing Neural Gas (GNG) is a suitable model because of its flexibility, rapid adaptation and excellent quality of representation. However, this type of learning is time-consuming, especially for high-dimensional input data. Since real applications often work under time constraints, it is necessary to adapt the learning process in order to complete it in a predefined time. This paper proposes a Graphics Processing Unit (GPU) parallel implementation of the GNG with Compute Unified Device Architecture (CUDA). In contrast to existing algorithms, the proposed GPU implementation allows the acceleration of the learning process keeping a good quality of representation. Comparative experiments using iterative, parallel and hybrid implementations are carried out to demonstrate the effectiveness of CUDA implementation. The results show that GNG learning with the proposed implementation achieves a speed-up of 6×$6\times$ compared with the single-threaded CPU implementation. GPU implementation has also been applied to a real application with time constraints: acceleration of 3D scene reconstruction for egomotion, in order to validate the proposal.     

A mesh optimization algorithm based on neural networks

We have developed a mesh simplification method called GNG3D which is able to produce high quality approximations of polygonal models. This method consists of two distinct phases: an optimization phase and a reconstruction phase. The optimization phase is developed by applying an extension algorithm of the growing neural gas model, which constitutes an unsupervised incremental clustering algorithm. The primary goal of this phase is to obtain a simplified set of vertices representing the best approximation of the original 3D object. In the reconstruction phase we use the information provided by the optimization algorithm to reconstruct the faces obtaining the optimized mesh as a result. We study the model theoretically, analyzing its main components, and experimentally, using for this purpose some 3D objects with different topologies. To evaluate the quality of approximations produced by the method proposed in this paper, three existing error measurements are used. The ability of the model to establish the number of vertices of the final simplified mesh is demonstrated in the examples.     

心内膜表面几何模型三维重建算法研究

心内膜三维表面重建是心内膜三维标测系统中的关键问题。为了满足实际应用需求, 根据采集到的散乱点云数据的特点, 提出了一种改进的泊松表面重建算法。在估计表面点云法向量的基础上, 对表面点云法向量进行法向量一致化处理, 有效地控制时间复杂度, 快速重建出平滑的心脏模型。针对泊松表面重建算法中构建MC曲面出现的二义性问题, 提出一种消除二义性的简化改进方法, 可以更加精确地获取模型逼真表面, 提高重建的速度和精度。同时, 可以根据医生的要求, 对重建出的模型实时修正, 满足临床应用。最后, 通过实验验证了算法的有效性和可行性。     

Three-dimensional (3D) reconstruction of structures and landscapes: A new point-and-line fusion method

The technique of three-dimensional (3D) reconstruction is widely used to develop infrastructure and landscape models to manage cities and assets better. Accurately reconstructing 3D structures (e.g., planes or lines) is a core step in rebuilding a model, especially within a built environment, where piece-wise planar/linear structures predominately prevail. As high-resolution images of large areas have become increasingly accessible, this paper develops an improved 3D reconstruction pipeline using the combination of point and line features. By introducing a dense reconstruction algorithm, which is an improved patch based stereo matching algorithm, this paper presents a robust approach that can be used to overcome the inaccuracies, integrity and reconstruction inefficiencies associated with point clouds. A 3D line extraction method is added to reconstruct accurate edges of buildings. The experimental results demonstrate that the proposed method visually improves the reconstruction effect of a 3D structure and a model's visualization.     

Three-dimensional motion and reconstruction of coronary arteries from biplane cineangiography

A new approach is described for reconstructing coronary arteries from two sequences of projection images. The estimation of motion is performed on three-dimensional line segments (or centrelines), and is based on a ‘predictionprojection-optimization’ loop. The method copes with time varying properties, deformations and superpositions of vessels. Experiments using simulated and real data have been carried out. and the results found to be robust over a full cycle of a human heart. Local and global kinetic features can then be derived to obtain a greater insight on the cardiac functional state     

用单数码相机实现物体表面的三维重建

根据由运动重建物体结构的原理,设计了一个简便易操作的三维重建系统,具体做法是:先用张氏标定法求得内参数矩阵,然后在两个不同的未知位置拍摄物体得到两幅图像,经立体匹配后,利用图像特征点的对应关系求解基本矩阵和本质矩阵,分解本质矩阵获得两个拍摄位置确定的摄像机运动参数(旋转矩阵和平移向量),进而求出相机在两个位置的投影矩阵,最后用三角法计算出物体表面特征点的三维坐标并在OpenGL中重建物体表面.和传统的立体视觉系统相比,本系统只需要一台数码相机和平面方格模板就可以实现三维重建,因此适用于普通相机用户.     

一种新型神经网络在自由曲面重构中的应用

以层次划分和模块化为思想基础,提出了一种新型神经网络模型对自由曲面进行重构,即基于径向基函数（RBF）神经网络的混合网络模型。先后运用减聚类方法、正交最小二乘法、最大似然法对网络进行有无监督的混合训练,旨在解决大样本集的简化建模和快速训练问题,提高混合网络输出精度。实验结果表明该网络模型使得曲面的拟合精度有了明显提高。     

A sum-of-product neural network (SOPNN)

This paper presents a sum-of-product neural network (SOPNN) structure. The SOPNN can learn to implement static mapping that multilayer neural networks and radial basis function networks normally perform. The output of the neural network has the sum-of-product form ∑Npi=1∏Nvj=1 fij (xj), where xj's are inputs, Nv is the number of inputs, fij( ) is a function generated through network training, and Np is the number of product terms. The function fij(xj) can be expressed as ∑kwijkBjk(xj), where Bjk( ) is a single-variable basis function and Wijk's are weight values. Linear memory arrays can be used to store the weights. If Bjk( ) is a Gaussian function, the new neural network degenerates to a Gaussian function network. This paper focuses on the use of overlapped rectangular pulses as the basis functions. With such basis functions, WijkBjk(xj) will equal either zero or Wijk, and the computation of fij(xj) becomes a simple addition of some retrieved Wijk's. The structure can be viewed as a basis function network with a flexible form for the basis functions. Learning can start with a small set of submodules and have new submodules added when it becomes necessary. The new neural network structure demonstrates excellent learning convergence characteristics and requires small memory space. It has merits over multilayer neural networks, radial basis function networks and CMAC in function approximation and mapping in high-dimensional input space. The technique has been tested for function approximation, prediction of a time series, learning control, and classification.     

Functional networks for B-spline surface reconstruction

Recently, a new extension of the standard neural networks, the so-called functional networks, has been described [E. Castillo, Functional networks, Neural Process. Lett. 7 (1998) 151–159]. This approach has been successfully applied to the reconstruction of a surface from a given set of 3D data points assumed to lie on unknown Bézier [A. Iglesias, A. Gálvez, Applying functional networks to CAGD: the tensor-product surface problem, in: D. Plemenos (Ed.), Proceedings of the International Conference on Computer Graphics and Artificial Intelligence, 3IA’2000, 2000, pp. 105–115; A. Iglesias, A. Gálvez, A new artificial intelligence paradigm for computer-aided geometric design, in: Artificial Intelligence and Symbolic Computation, J.A. Campbell, E. Roanes-Lozano (Eds.), Lectures Notes in Artificial Intelligence, Berlin, Heidelberg, Springer-Verlag, vol. 1930, 2001, pp. 200–213] and B-spline tensor-product surfaces [A. Iglesias, A. Gálvez, Applying functional networks to fit data points from B-spline surfaces, in: H.H.S. Ip, N. Magnenat-Thalmann, R.W.H. Lau, T.S. Chua (Eds.), Proceedings of the Computer Graphics International, CGI’2001, IEEE Computer Society Press, Los Alamitos, CA, 2001, pp. 329–332]. In both cases the sets of data were fitted using Bézier surfaces. However, in general, the Bézier scheme is no longer used for practical applications. In this paper, the use of B-spline surfaces (by far the most common family of surfaces in surface modeling and industry) for the surface reconstruction problem is proposed instead. The performance of this method is discussed by means of several illustrative examples. A careful analysis of the errors makes it possible to determine the number of B-spline surface fitting control points that best fit the data points. This analysis also includes the use of two sets of data (the training and the testing data) to check for overfitting, which does not occur here.     

Dynamic numerical reconstruction of a fungal biofiltration system using differential neural network

Biofiltration is an economical and environmentally friendly process to eliminate air pollutants. Results obtained by different authors showed the enhanced performance of the fungal biofiltering systems. Consequently, there is a necessity to develop methodologies not only to design more efficient reactors but to control the reaction behavior under different conditions: pollutants feeding, air flows, humidity and biomass production. In this study, a continuous neural network observer was designed to predict the toluene vapors elimination capacity (EC) in a fungal biofilter. The observer uses the carbon dioxide (CO₂) production and the pressure drop (DP) (on line measurements) as input information. The differential neural network observer proved to be a useful tool to reconstruct the immeasurable on-line variable (EC). The observer was successfully tested under different reaction conditions proving the robustness of estimation process. This software sensor may be helpful to derive adaptive control functions optimizing the biofilter reaction development.     

Estimation of cutting forces and surface roughness for hard turning using neural networks

Metal cutting mechanics is quite complicated and it is very difficult to develop a comprehensive model which involves all cutting parameters affecting machining variables. In this study, machining variables such as cutting forces and surface roughness are measured during turning at different cutting parameters such as approaching angle, speed, feed and depth of cut. The data obtained by experimentation is analyzed and used to construct model using neural networks. The model obtained is then tested with the experimental data and results are indicated.     

Assigning discounts in a marketing campaign by using reinforcement learning and neural networks

In this work, RL is used to find an optimal policy for a marketing campaign. Data show a complex characterization of state and action spaces. Two approaches are proposed to circumvent this problem. The first approach is based on the self-organizing map (SOM), which is used to aggregate states. The second approach uses a multilayer perceptron (MLP) to carry out a regression of the action-value function. The results indicate that both approaches can improve a targeted marketing campaign. Moreover, the SOM approach allows an intuitive interpretation of the results, and the MLP approach yields robust results with generalization capabilities.     

Stable nonlinear adaptive controller for an autonomous underwater vehicle using neural networks

In general, the dynamics of autonomous underwater vehicles (AUVs) are highly nonlinear and their hydrodynamic coefficients vary with different operating conditions. For this reason, high performance control system for an AUV usually should have the capacities of learning and adaptation to the time-varying dynamics of the vehicle. In this article, we present a robust adaptive nonlinear control scheme for an AUV, where a linearly parameterized neural network (LPNN) is introduced to approximate the uncertainties of the vehicle's dynamics, and the basis function vector of the network is constructed according to the vehicle's physical properties. The proposed control scheme can guarantee that all of the signals in the closed-loop system are uniformly ultimately bounded (UUB). Numerical simulation studies are performed to illustrate the effectiveness of the proposed control scheme.     

3D reconstruction using silhouettes from unordered viewpoints

In this paper, we present a novel approach for reconstructing an object surface from its silhouettes. The proposed approach directly estimates the differential structure of the surface, and results in a higher accuracy than existing volumetric approaches for object reconstruction. Compared with other existing differential approaches, our approach produces relatively complete 3D models similar to volumetric approaches, with the topology conforming to what is observed from the silhouettes. In addition, the method neither assumes nor depends on the spatial order of viewpoints. Experimental results on both synthetic and real world data are presented, and comparison is made with other existing approaches to demonstrate the superiority of the proposed approach.     

一种快速有效实现三维实体重建的算法

从图的数组表示法这一基本表示方法,作为基点出发,将点、线、面、面环等信息用数组形式存储,从数组元素出发逐步实现了基于三视图的三维重建。实践表明,该方法充分利用数组形式的有序、对应、直接等特点,大大提高了三维重建的效率,减少了传统方法庞大的搜索空间和降低了时间复杂度。     

A pointwise smooth surface stereo reconstruction algorithm without correspondences

This paper describes an algorithm for 3D reconstruction of a smooth surface with a relatively dense set of self-similar point features from two calibrated views. We bypass the usual correspondence problem by triangulating a point in space from all pairs of features satisfying the epipolar constraint. The surface is then extracted from the resulting point cloud by taking advantage of the statistical and geometric properties of the point distribution on the surface. Results are presented for computer simulations and for a laboratory experiment on a silicon gel phantom used in a breast cancer screening project.     

面向点云三维重建的空间感知对抗神经网络

为了解决由原始点云数据局部密度稀疏、不均匀或者法向量错误等制约因素引起的重建网格质量问题,利用对抗神经网络中权重共享的特性和对抗的训练过程,提出一种基于对抗网络的点云三维重建方法。首先,利用预测器对网格模型边的偏移量进行预测,从而得到每一个顶点的位移,并进行拓扑保持的顶点重定向,得到新的网格模型。然后,利用判别器中的点云分类器,提取原始点云数据和网格模型表面采样点集的高维特征,并基于高维特征进行空间感知的判别,用于区分原始点云与采样点集数据。最后,使用对抗的训练方式将预测器与判别器的输出数据关联起来,通过多次迭代优化网络模型,从而得到满足点云空间特征的三维网格模型。在不同的点云数据集上进行实验,并使用MeshLab软件进行效果展示,结果表明,该方法能够重建出满足点云空间信息的三维网格模型,同时能够解决粗劣的点云数据引起的网格质量问题。     

三维散乱点云快速曲面重建算法

提出了一种基于Delaunay三角剖分的三维散乱点云快速曲面重建算法。算法首先计算点云的Delaunay三角剖分, 从Delaunay四面体提取初始三角网格, 根据Voronoi体元的特征构造优先队列并生成种子三角网格, 然后通过区域生长的方式进行流形提取。实验结果表明, 该算法可以高效、稳定地重构具有复杂拓扑结构、非封闭曲面甚至是非均匀采样的点云数据。与传统的基于Delaunay的方法比较, 该算法仅需要进行一次Delaunay三角剖分, 无须极点的计算, 因此算法的重构速度快。     

基于VTK的三维点云曲面重建研究

针对三维点云数据重建效率低、不能实时交互等问题,利用鲁棒性强的Power Crust算法和三维可视化类库Visualization Toolkit (VTK)的良好并行机制与强大的图像处理能力,实现了三维点云数据曲面快速重建.该算法使用Power Crust对三维点云进行曲面重建,接着对得到的网格进行线性调整、简化和平滑,最后引入VTK进行渲染、绘制、显示,并实时交互.实验结果表明,该算法可以加快散乱点云数据的重建速度,较好地保持了点云数据的拓扑结构,提高了曲面重建的精确性和鲁棒性,且交互性强,适合实时处理.