GPGPU implementation of growing neural gas: Application to 3D scene reconstruction

Self-organising neural models have the ability to provide a good representation of the input space. In particular the Growing Neural Gas (GNG) is a suitable model because of its flexibility, rapid adaptation and excellent quality of representation. However, this type of learning is time-consuming, especially for high-dimensional input data. Since real applications often work under time constraints, it is necessary to adapt the learning process in order to complete it in a predefined time. This paper proposes a Graphics Processing Unit (GPU) parallel implementation of the GNG with Compute Unified Device Architecture (CUDA). In contrast to existing algorithms, the proposed GPU implementation allows the acceleration of the learning process keeping a good quality of representation. Comparative experiments using iterative, parallel and hybrid implementations are carried out to demonstrate the effectiveness of CUDA implementation. The results show that GNG learning with the proposed implementation achieves a speed-up of 6×$6\times$ compared with the single-threaded CPU implementation. GPU implementation has also been applied to a real application with time constraints: acceleration of 3D scene reconstruction for egomotion, in order to validate the proposal.     

A mesh optimization algorithm based on neural networks

We have developed a mesh simplification method called GNG3D which is able to produce high quality approximations of polygonal models. This method consists of two distinct phases: an optimization phase and a reconstruction phase. The optimization phase is developed by applying an extension algorithm of the growing neural gas model, which constitutes an unsupervised incremental clustering algorithm. The primary goal of this phase is to obtain a simplified set of vertices representing the best approximation of the original 3D object. In the reconstruction phase we use the information provided by the optimization algorithm to reconstruct the faces obtaining the optimized mesh as a result. We study the model theoretically, analyzing its main components, and experimentally, using for this purpose some 3D objects with different topologies. To evaluate the quality of approximations produced by the method proposed in this paper, three existing error measurements are used. The ability of the model to establish the number of vertices of the final simplified mesh is demonstrated in the examples.     

Three-dimensional motion and reconstruction of coronary arteries from biplane cineangiography

A new approach is described for reconstructing coronary arteries from two sequences of projection images. The estimation of motion is performed on three-dimensional line segments (or centrelines), and is based on a ‘predictionprojection-optimization’ loop. The method copes with time varying properties, deformations and superpositions of vessels. Experiments using simulated and real data have been carried out. and the results found to be robust over a full cycle of a human heart. Local and global kinetic features can then be derived to obtain a greater insight on the cardiac functional state     

用单数码相机实现物体表面的三维重建

根据由运动重建物体结构的原理,设计了一个简便易操作的三维重建系统,具体做法是:先用张氏标定法求得内参数矩阵,然后在两个不同的未知位置拍摄物体得到两幅图像,经立体匹配后,利用图像特征点的对应关系求解基本矩阵和本质矩阵,分解本质矩阵获得两个拍摄位置确定的摄像机运动参数(旋转矩阵和平移向量),进而求出相机在两个位置的投影矩阵,最后用三角法计算出物体表面特征点的三维坐标并在OpenGL中重建物体表面.和传统的立体视觉系统相比,本系统只需要一台数码相机和平面方格模板就可以实现三维重建,因此适用于普通相机用户.     

A sum-of-product neural network (SOPNN)

This paper presents a sum-of-product neural network (SOPNN) structure. The SOPNN can learn to implement static mapping that multilayer neural networks and radial basis function networks normally perform. The output of the neural network has the sum-of-product form ∑Npi=1∏Nvj=1 fij (xj), where xj's are inputs, Nv is the number of inputs, fij( ) is a function generated through network training, and Np is the number of product terms. The function fij(xj) can be expressed as ∑kwijkBjk(xj), where Bjk( ) is a single-variable basis function and Wijk's are weight values. Linear memory arrays can be used to store the weights. If Bjk( ) is a Gaussian function, the new neural network degenerates to a Gaussian function network. This paper focuses on the use of overlapped rectangular pulses as the basis functions. With such basis functions, WijkBjk(xj) will equal either zero or Wijk, and the computation of fij(xj) becomes a simple addition of some retrieved Wijk's. The structure can be viewed as a basis function network with a flexible form for the basis functions. Learning can start with a small set of submodules and have new submodules added when it becomes necessary. The new neural network structure demonstrates excellent learning convergence characteristics and requires small memory space. It has merits over multilayer neural networks, radial basis function networks and CMAC in function approximation and mapping in high-dimensional input space. The technique has been tested for function approximation, prediction of a time series, learning control, and classification.     

Functional networks for B-spline surface reconstruction

Recently, a new extension of the standard neural networks, the so-called functional networks, has been described [E. Castillo, Functional networks, Neural Process. Lett. 7 (1998) 151–159]. This approach has been successfully applied to the reconstruction of a surface from a given set of 3D data points assumed to lie on unknown Bézier [A. Iglesias, A. Gálvez, Applying functional networks to CAGD: the tensor-product surface problem, in: D. Plemenos (Ed.), Proceedings of the International Conference on Computer Graphics and Artificial Intelligence, 3IA’2000, 2000, pp. 105–115; A. Iglesias, A. Gálvez, A new artificial intelligence paradigm for computer-aided geometric design, in: Artificial Intelligence and Symbolic Computation, J.A. Campbell, E. Roanes-Lozano (Eds.), Lectures Notes in Artificial Intelligence, Berlin, Heidelberg, Springer-Verlag, vol. 1930, 2001, pp. 200–213] and B-spline tensor-product surfaces [A. Iglesias, A. Gálvez, Applying functional networks to fit data points from B-spline surfaces, in: H.H.S. Ip, N. Magnenat-Thalmann, R.W.H. Lau, T.S. Chua (Eds.), Proceedings of the Computer Graphics International, CGI’2001, IEEE Computer Society Press, Los Alamitos, CA, 2001, pp. 329–332]. In both cases the sets of data were fitted using Bézier surfaces. However, in general, the Bézier scheme is no longer used for practical applications. In this paper, the use of B-spline surfaces (by far the most common family of surfaces in surface modeling and industry) for the surface reconstruction problem is proposed instead. The performance of this method is discussed by means of several illustrative examples. A careful analysis of the errors makes it possible to determine the number of B-spline surface fitting control points that best fit the data points. This analysis also includes the use of two sets of data (the training and the testing data) to check for overfitting, which does not occur here.     

Dynamic numerical reconstruction of a fungal biofiltration system using differential neural network

Biofiltration is an economical and environmentally friendly process to eliminate air pollutants. Results obtained by different authors showed the enhanced performance of the fungal biofiltering systems. Consequently, there is a necessity to develop methodologies not only to design more efficient reactors but to control the reaction behavior under different conditions: pollutants feeding, air flows, humidity and biomass production. In this study, a continuous neural network observer was designed to predict the toluene vapors elimination capacity (EC) in a fungal biofilter. The observer uses the carbon dioxide (CO₂) production and the pressure drop (DP) (on line measurements) as input information. The differential neural network observer proved to be a useful tool to reconstruct the immeasurable on-line variable (EC). The observer was successfully tested under different reaction conditions proving the robustness of estimation process. This software sensor may be helpful to derive adaptive control functions optimizing the biofilter reaction development.     

Estimation of cutting forces and surface roughness for hard turning using neural networks

Metal cutting mechanics is quite complicated and it is very difficult to develop a comprehensive model which involves all cutting parameters affecting machining variables. In this study, machining variables such as cutting forces and surface roughness are measured during turning at different cutting parameters such as approaching angle, speed, feed and depth of cut. The data obtained by experimentation is analyzed and used to construct model using neural networks. The model obtained is then tested with the experimental data and results are indicated.     

Assigning discounts in a marketing campaign by using reinforcement learning and neural networks

In this work, RL is used to find an optimal policy for a marketing campaign. Data show a complex characterization of state and action spaces. Two approaches are proposed to circumvent this problem. The first approach is based on the self-organizing map (SOM), which is used to aggregate states. The second approach uses a multilayer perceptron (MLP) to carry out a regression of the action-value function. The results indicate that both approaches can improve a targeted marketing campaign. Moreover, the SOM approach allows an intuitive interpretation of the results, and the MLP approach yields robust results with generalization capabilities.     

3D reconstruction using silhouettes from unordered viewpoints

In this paper, we present a novel approach for reconstructing an object surface from its silhouettes. The proposed approach directly estimates the differential structure of the surface, and results in a higher accuracy than existing volumetric approaches for object reconstruction. Compared with other existing differential approaches, our approach produces relatively complete 3D models similar to volumetric approaches, with the topology conforming to what is observed from the silhouettes. In addition, the method neither assumes nor depends on the spatial order of viewpoints. Experimental results on both synthetic and real world data are presented, and comparison is made with other existing approaches to demonstrate the superiority of the proposed approach.     

一种快速有效实现三维实体重建的算法

从图的数组表示法这一基本表示方法,作为基点出发,将点、线、面、面环等信息用数组形式存储,从数组元素出发逐步实现了基于三视图的三维重建。实践表明,该方法充分利用数组形式的有序、对应、直接等特点,大大提高了三维重建的效率,减少了传统方法庞大的搜索空间和降低了时间复杂度。     

A pointwise smooth surface stereo reconstruction algorithm without correspondences

This paper describes an algorithm for 3D reconstruction of a smooth surface with a relatively dense set of self-similar point features from two calibrated views. We bypass the usual correspondence problem by triangulating a point in space from all pairs of features satisfying the epipolar constraint. The surface is then extracted from the resulting point cloud by taking advantage of the statistical and geometric properties of the point distribution on the surface. Results are presented for computer simulations and for a laboratory experiment on a silicon gel phantom used in a breast cancer screening project.     

基于VTK的三维点云曲面重建研究

针对三维点云数据重建效率低、不能实时交互等问题,利用鲁棒性强的Power Crust算法和三维可视化类库Visualization Toolkit (VTK)的良好并行机制与强大的图像处理能力,实现了三维点云数据曲面快速重建.该算法使用Power Crust对三维点云进行曲面重建,接着对得到的网格进行线性调整、简化和平滑,最后引入VTK进行渲染、绘制、显示,并实时交互.实验结果表明,该算法可以加快散乱点云数据的重建速度,较好地保持了点云数据的拓扑结构,提高了曲面重建的精确性和鲁棒性,且交互性强,适合实时处理.     

3D reconstruction of free-formed line-like objects using NURBS representation

3D reconstruction of arbitrary free-formed objects is an important and challenging problem in computer vision. In this paper, we first discuss the importance of primitive selection in 3D reconstruction. Subsequently, a theorem, which reveals the perspective invariance of NURBS, is shown, making it a good choice as primitive in 3D reconstruction. Consequently, based on this theorem a new paradigm of free-formed line-like object reconstruction using NURBS as primitives is proposed. Furthermore, an approach for determining weights for 3D NURBS is presented, and the width effect of curved line-like objects is analyzed. Finally, experiments with line-like objects and machine part demonstrate the feasibility of our approach and prove the superiority of our approach over the point- or segment-based approaches as well as the B-spline-based reconstruction approach in terms of robustness and accuracy.     

Adaptive supervisory WCMAC neural network controller (SWC) for nonlinear systems

This paper proposes a wavelet-based cerebellar model arithmetic controller neural network (called WCMAC) and develops an adaptive supervisory WCMAC control (SWC) scheme for nonlinear uncertain systems. The WCMAC is modified from the traditional CMAC for obtaining high approximation accuracy and convergent rate using the advantages of wavelet functions and fuzzy TSK-model. For nonlinear uncertain systems, a PD-type WCMAC controller with filter is constructed to approximate an ideal control signal. The corresponding adaptive supervisory controller is used to recover the residual of approximation error. Finally, the adaptive SWC scheme is applied to chaotic system identification and control including Mackey–Glass time-series prediction, control of inverted pendulum system, and control of Chua circuit system. These demonstrate the effectiveness of our adaptive SWC approach for nonlinear uncertain systems.     

Video shot motion characterization based on hierarchical overlapped growing neural gas networks

Automatic video shot motion characterization is an important step in video indexing and retrieval after temporal video segmentation. This paper describes a hierarchical overlapped architecture (HOGNG) based upon the growing neural gas (GNG) network [7] to perform this task. The proposed architecture combines the unsupervised and supervised learning schemes in GNG. As higher-level GNGs overlap, the final classification is obtained by fusing the individual classifications generated by the top-level overlapping GNGs. In addition, we employ prefiltering and postfiltering for improving the classification accuracy. Experimental results are presented to show the good classification accuracy of the proposed algorithm on real MPEG video sequences.P.N. Suganthan: Correspondence to     

Towards a real-time 3D shape reconstruction using a structured light system

This paper deals with 3D shape reconstruction using a structured light system (SLS) which projects a matrix of laser rays onto the scene to be analyzed. The intrinsic problem of such a system is the correspondence problem solving, which in this particular case amounts to matching up the imaged spots and the originating laser rays. In this paper, we propose a method for automatically obtaining configurations of the system (COS) (i.e. the relative positions of the camera, laser projector, and measuring scene) that permit to achieve a direct and unambiguous correspondence. After, we propose a splitting cell algorithm, which efficiently performs a real-time correspondence procedure. Experimental results obtained from both simulated and real data demonstrate that our method provides our SLS with possibilities for real-time applications.     

Capturing tree crown formation through implicit surface reconstruction using airborne lidar data

Forest structure data derived from lidar is being used in forest science and management for inventory analysis, biomass estimation, and wildlife habitat analysis. Regression analysis dominated previous approaches to the derivation of tree stem and crown parameters from lidar. The regression model for tree parameters is locally applied based on vertical lidar point density, the tree species involved, and stand structure in the specific research area. The results of this approach, therefore, are location-specific, limiting its applicability to other areas. For a more widely applicable approach to derive tree parameters, we developed an innovative method called ‘wrapped surface reconstruction’ that employs radial basis functions and an isosurface. Utilizing computer graphics, we capture the exact shape of an irregular tree crown of various tree species based on the lidar point cloud and visualize their exact crown formation in three-dimensional space. To validate the tree parameters given by our wrapped surface approach, survey-grade equipment (a total station) was used to measure the crown shape. Four vantage points were established for each of 55 trees to capture whole-tree crown profiles georeferenced with post-processed differential GPS points. The observed tree profiles were linearly interpolated to estimate crown volume. These fieldwork-generated profiles were compared with the wrapped surface to assess goodness of fit. For coniferous trees, the following tree crown parameters derived by the wrapped surface method were highly correlated (p < 0.05) with the total station-derived measurements: tree height (R² = 0.95), crown width (R² = 0.80), live crown base (R² = 0.92), height of the lowest branch (R² = 0.72), and crown volume (R² = 0.84). For deciduous trees, wrapped surface-derived parameters of tree height (R² = 0.96), crown width (R² = 0.75), live crown base (R² = 0.53), height of the lowest branch (R² = 0.51), and crown volume (R² = 0.89) were correlated with the total station-derived measurements. The wrapped surface technique is less susceptible to errors in estimation of tree parameters because of exact interpolation using the radial basis functions. The effect of diminished energy return causes the low correlation for lowest branches in deciduous trees (R² = 0.51), even though leaf-off lidar data was used. The wrapped surface provides fast and automated detection of micro-scale tree parameters for specific applications in areas such as tree physiology, fire modeling, and forest inventory.