Dynamics on higher-order networks: a review

Network science has evolved into an indispensable platform for studying complex systems. But recent research has identified limits of classical networks, where links connect pairs of nodes, to comprehensively describe group interactions. Higher-order networks, where a link can connect more than two nodes, have therefore emerged as a new frontier in network science. Since group interactions are common in social, biological and technological systems, higher-order networks have recently led to important new discoveries across many fields of research. Here, we review these works, focusing in particular on the novel aspects of the dynamics that emerges on higher-order networks. We cover a variety of dynamical processes that have thus far been studied, including different synchronization phenomena, contagion processes, the evolution of cooperation and consensus formation. We also outline open challenges and promising directions for future research.     

Variance decomposition-based sensitivity analysis via neural networks

This paper illustrates a method for efficiently performing multiparametric sensitivity analyses of the reliability model of a given system. These analyses are of great importance for the identification of critical components in highly hazardous plants, such as the nuclear or chemical ones, thus providing significant insights for their risk-based design and management. The technique used to quantify the importance of a component parameter with respect to the system model is based on a classical decomposition of the variance. When the model of the system is realistically complicated (e.g. by aging, stand-by, maintenance, etc.), its analytical evaluation soon becomes impractical and one is better off resorting to Monte Carlo simulation techniques which, however, could be computationally burdensome. Therefore, since the variance decomposition method requires a large number of system evaluations, each one to be performed by Monte Carlo, the need arises for possibly substituting the Monte Carlo simulation model with a fast, approximated, algorithm. Here we investigate an approach which makes use of neural networks appropriately trained on the results of a Monte Carlo system reliability/availability evaluation to quickly provide with reasonable approximation, the values of the quantities of interest for the sensitivity analyses. The work was a joint effort between the Department of Nuclear Engineering of the Polytechnic of Milan, Italy, and the Institute for Systems, Informatics and Safety, Nuclear Safety Unit of the Joint Research Centre in Ispra, Italy which sponsored the project.     

Variable selection for neural networks in multivariate calibration

The problem of variable selection for neural network modeling is discussed in this paper. Two methods that gave the best results in a previous comparative study are presented. One of these methods is a modified version of the Hinton diagrams, the other method is based on saliency estimation and is part of the Optimal Brain Surgeon algorithm for pruning unimportant weights in a neural network. We also propose two new methods, based on the estimation of the contribution of each input variable to the variance of the predicted response. These new methods are designed for situations where input variables are orthogonal, such as the PC scores often used in multivariate calibration. The four methods are tested on synthetic examples, and on real industrial data sets for multivariate calibration. The main characteristics of each method are discussed. In particular, we underline the strong theoretical and experimental limitations of methods like the modified Hinton diagrams, based on weight magnitude estimation. We also demonstrate that although the saliency estimation approach is theoretically more stringent, it gives unstable results on repeated trials. The advantage of the two variance-based approaches is that they are much less dependent on the initial weight randomization than the two other methods, and therefore, the results they produce are more stable and reliable.     

An analysis of the equilibria of neural networks with linear interconnections

In this paper, we analyse the equilibria of neural networks which consist of a set of sigmoid nonlinearities with linear interconnections,without assuming that the interconnections are symmetric or that there are no self-interactions. By eliminating these assumptions, we are able to study the effects of imperfect implementation on the behaviour of Hopfield networks. If one views the neural network as evolving on the openn-dimensional hypercubeH = (0, 1) ⁿ , we have the following conclusions as the neural characteristics become steeper and steeper: (i) There is at most one equilibrium in any compact subset ofH, and under mild assumptions this equilibrium is unstable. In fact, the dimension of the stable manifold of this equilibrium is the same as the number of eigenvalues of the interconnection matrix with negative real parts. (ii) There might be some equilibria in the faces ofH, and under mild conditions these are always unstable. Moreover, it is easy to compute the dimension of the stable manifold of each such equilibrium. (iii) A systematic procedure is given for determining which corners of the hypercubeH contain equilibria, and it is shown that all equilibria in the corners ofH are asymptotically stable.     

A model updating strategy of non‐linear vibrating structures

The objective of this paper is to present a model updating strategy of non‐linear vibrating structures. Because modal analysis is no longer helpful in non‐linear structural dynamics, a special attention is devoted to the features extracted from the proper orthogonal decomposition and one of its non‐linear generalizations based on auto‐associative neural networks. The efficiency of the proposed procedure is illustrated using simulated data from a three‐dimensional portal frame. Copyright © 2004 John Wiley & Sons, Ltd.     

Flow curve prediction of an Al-MMC under hot working conditions using neural networks

The plastic flow behaviour of a particle-reinforced aluminium alloy matrix composite (AA2618 + Al₂O_3p) was studied by analysing the results of hot compression tests carried out in extended ranges of temperature and strain rate, typical of hot working operations. In general, for a given temperature and strain rate, the flow curves exhibit a peak, at relatively low strains, followed by flow softening; for a constant strain, the flow stress increases with increasing strain rate and decreasing temperature. The experimental data were used as an input for training artificial neural networks in order to predict the flow curves of the composite investigated. The comparison of the predicted stress–strain curves with the ones obtained by experimental testing, under conditions different from those used for the training stage, has proven the prediction generalisation capability of the artificial neural network-based models.     

Machine and component residual life estimation through the application of neural networks

This paper concerns the use of neural networks for predicting the residual life of machines and components. In addition, the advantage of using condition-monitoring data to enhance the predictive capability of these neural networks was also investigated. A number of neural network variations were trained and tested with the data of two different reliability-related datasets. The first dataset represents the renewal case where the failed unit is repaired and restored to a good-as-new condition. Data were collected in the laboratory by subjecting a series of similar test pieces to fatigue loading with a hydraulic actuator. The average prediction error of the various neural networks being compared varied from 431 to 841 s on this dataset, where test pieces had a characteristic life of 8971 s. The second dataset were collected from a group of pumps used to circulate a water and magnetite solution within a plant. The data therefore originated from a repaired system affected by reliability degradation. When optimized, the multi-layer perceptron neural networks trained with the Levenberg-Marquardt algorithm and the general regression neural network produced a sum-of-squares error within 11.1% of each other for the renewal dataset. The small number of inputs and poorly mapped input space on the second dataset meant that much larger errors were recorded on some of the test data. The potential for using neural networks for residual life prediction and the advantage of incorporating condition-based data into the model was nevertheless proven for both examples.     

Predicting the effectiveness of blast wall barriers using neural networks

Blast damage assessment of buildings and structural elements requires an accurate prediction of the blast loads in terms of the peak pressures and impulses. Blast loadings on structures have typically been evaluated using empirical relationships. These relationships assume that there are no obstacles between the explosive device and the target. If a blast barrier is used to protect personnel or a structure behind it, the actual blast loading environment will be significantly reduced for some distance behind the barrier. This paper is concerned with an accurate prediction of the area of effectiveness behind the barrier using experimental data and a neural network-based model. To train and validate the neural network, a database is developed through a series of measurements of the blast environment behind the barrier. The principal parameters controlling the blast environment, such as wall height, distance behind the wall, height above ground, and standoff distance are used as the training input data. Peak overpressure and peak scaled impulse are used as the outputs in the neural network configuration. The trained and validated neural network is used to develop contour plots of overpressure and impulse adjustment factors to simplify the process of predicting the effectiveness of blast barriers. The developed model is also deployed in a stand-alone application that is used as a fast-running predictive tool for the blast overpressures and impulses behind the wall.     

Analysis of one-dimensional stochastic finite elements using neural networks

This paper explores the applicability of neural networks for analyzing the uncertainty spread of structural responses under the presence of one-dimensional random fields. Specifically, the neural network is intended to be a partial surrogate of the structural model needed in a Monte Carlo simulation, due to its associative memory properties. The network is trained with some pairs of input and output data obtained by some Monte Carlo simulations and then used in substitution of the finite element solver. In order to minimize the size of the networks, and hence the number of training pairs, the Karhunen–Loéve decomposition is applied as an optimal feature extraction tool. The Monte Carlo samples for training and validation are also generated using this decomposition. The Nyström technique is employed for the numerical solution of the Fredholm integral equation. The radial basis function (RBF) network was selected as the neural device for learning the input/output relationship due to its high accuracy and fast training speed. The analysis shows that this approach constitutes a promising method for stochastic finite element analysis inasmuch as the error with respect to the Monte Carlo simulation is negligible.     

建立CRT色度变换的神经网络模型

针对7点LOG空间分布方案的343个训练样本,提出了采用2个隐层和少节点网络结构的方案,并用拟牛顿法训练神经网络模型。采用10点LOG空间分布方案中不同于训练样本的216个检验样本实时监控训练过程,以避免出现“过训练”现象,从而求得全局极小点邻域内的可行解,建立从CRT的R,G,B到CIE的X,Y,Z色度空间变换的BP模型。实例计算表明,该模型在收敛性、训练时间和泛化能力等方面均明显优于采用3～4个隐层方案的模型;模型的色差平均转换精度接近0.60个CIELUV色差单位,标准离差为0.57个色差单位,而4个隐层方案模型的色差平均精度和标准离差分别为1.53和0.77个CIELUV色差单位。     

Using ensemble and metaheuristics learning principles with artificial neural networks to improve due date prediction performance

One of the common and important problems in production scheduling is to quote an attractive but attainable due date for an arriving customer order. Among a wide variety of prediction methods proposed to improve due date quotation (DDQ) accuracy, artificial neural networks (ANN) are considered the most effective because of their flexible non-linear and interaction effects modelling capability. In spite of this growing use of ANNs in a DDQ context, ANNs have several intrinsic shortcomings such as instability, bias and variance problems that undermine their accuracy. In this paper, we develop an enhanced ANN-based DDQ model using machine learning, evolutionary and metaheuristics learning concepts. Computational experiments suggest that the proposed model outperforms the conventional ANN-based DDQ method under different shop environments and different training data sizes.     

Performance-based design and seismic reliability analysis using designed experiments and neural networks

Seismic design involves many uncertainties that arise from the earthquake motions, structural geometries, material properties, and analytical models. Taking into account all major uncertainties, reliability analysis is applied to estimate probability of failure in each of a set of performance requirements. The probability estimation is best conducted through Monte Carlo simulations with variance reduction techniques. However, this may involve many performance function evaluations, each requiring a non-linear dynamic analysis, which may be very computationally demanding. In order to improve computational efficiency, this paper explores Design of Computer Experiments and Neural Networks for representation of structural behavior. The neural networks are directly employed for reliability assessment and design optimization. Performance-based seismic design is formulated as an optimization problem, with design parameters optimally calculated. Two case studies are presented to demonstrate efficiency and applicability of the methodology: a bridge bent with or without seismic isolation and a steel pipe pile foundation.     

A modified nonlocal couple stress-based beam model for vibration analysis of higher-order FG nanobeams

In the present paper, nonlocal couple stress theory is developed to investigate free vibration characteristics of functionally graded (FG) nanobeams considering exact position of neutral axis. The theory introduces two parameters based on nonlocal elasticity theory and modified couple stress theory to capture the size effects much accurately. Therefore, a nonlocal stress field parameter and a material length scale parameter are used to involve both stiffness-softening and stiffness-hardening effects on responses of FG nanobeams. The FG nanobeam is modeled via a higher-order refined beam theory in which shear deformation effect is verified needless of shear correction factor. A power-law distribution is used to describe the graded material properties. The governing equations and the related boundary conditions are derived by Hamilton's principle and they are solved applying Galerkin's method, which satisfies various boundary conditions. A comparison study is performed to verify the present formulation with the provided data in the literature and a good agreement is observed. The parametric study covered in this paper includes several parameters, such as nonlocal and length scale parameters, power-law exponent, slenderness ratio, shear deformation, and various boundary conditions on natural frequencies of FG nanobeams in detail.     

Use of artificial neural networks to predict thickness and optical constants of thin films from reflectance data

Artificial neural networks and the Levenberg–Marquardt algorithm are combined to calculate the thickness and refractive index of thin films from spectroscopic reflectometry data. Two examples will be discussed, the first is a measurement of thickness and index of transparent films on silicon, and the second is a measurement of three thicknesses and index of poly-silicon in a rough poly-silicon on oxide stack. A neural network is a set of simple, highly interconnected processing elements imitating the activity of the brain, which are capable of learning information presented to them. Reflectometry has been used by the semiconductor industry to measure thin film thickness for decades. Modeling the optical constants of a film in the visible region with a Cauchy dispersion model allows the determination of both thickness and refractive index of most transparent thin films from reflectance data. The use of an alloy interpolation model for the optical constants of poly-silicon allows the determination of thicknesses and poly optical constants. In this work artificial neural networks are used to obtain good initial estimates for thickness and dispersion model parameters, these estimates are then used as the starting point for the Levenberg–Marquardt algorithm which converges to the final solution in a few iterations. These measurement programs were implemented on a Nanometrics NanoSpec 8000XSE.     

On reaction network implementations of neural networks

This paper is concerned with the utilization of deterministically modelled chemical reaction networks for the implementation of (feed-forward) neural networks. We develop a general mathematical framework and prove that the ordinary differential equations (ODEs) associated with certain reaction network implementations of neural networks have desirable properties including (i) existence of unique positive fixed points that are smooth in the parameters of the model (necessary for gradient descent) and (ii) fast convergence to the fixed point regardless of initial condition (necessary for efficient implementation). We do so by first making a connection between neural networks and fixed points for systems of ODEs, and then by constructing reaction networks with the correct associated set of ODEs. We demonstrate the theory by constructing a reaction network that implements a neural network with a smoothed ReLU activation function, though we also demonstrate how to generalize the construction to allow for other activation functions (each with the desirable properties listed previously). As there are multiple types of ‘networks’ used in this paper, we also give a careful introduction to both reaction networks and neural networks, in order to disambiguate the overlapping vocabulary in the two settings and to clearly highlight the role of each network’s properties.     

Assessment of the effect of existing corrosion on the tensile behaviour of magnesium alloy AZ31 using neural networks

A concept has been devised to assess the effect of existing corrosion damage on the residual tensile properties of structural alloys and applied for the magnesium alloy AZ31. The concept based on the use of a radial basis function neural network. An extensive experimental investigation, including metallographic corrosion characterization and mechanical testing of pre-corroded AZ31 magnesium alloy specimens, was carried out to derive the necessary data for the training and the prediction module of the developed neural network model. The proposed concept was exploited to successfully predict: the gradual tensile property degradation of the alloy AZ31 to the results of gradually increasing corrosion damage with increasing corrosion exposure.     

采用神经网络控制的超精密平面定位

设计了一种基于激光莫尔信号的超精密平面定位装置,该装置可实现高精度位置检测及X-Y-θ 三自由度的全自动精密平面定位。激光莫尔信号传感器为相位相差 180°的两组衍射光栅,构成差动光栅技术,可有效提高位置检测信号灵敏度及定位精度。针对系统存在非线性,精确控制模型难以建立的缺陷,提出基于 RBF 神经网络的精密定位控制,其控制响应快、稳定性好、鲁棒性强,可有效改善控制质量,提高定位速度。实验结果表明,基于激光莫尔信号的精密定位装置可获得亚微米级的平面定位精度。     

A new directional simulation method for system reliability. Part II: application of neural networks

A challenge in directional importance sampling is in identifying the location and the shape of the importance sampling density function when a realistic limit state for a structural system is considered in a finite element-supported reliability analysis. Deterministic point refinement schemes, previously studied in place of directional importance sampling, can be improved by prior knowledge of the limit state. This paper introduces two types of neural networks that identify the location and shape of the limit state quickly and thus facilitate directional simulation-based reliability assessment using the deterministic Fekete point sets introduced in the companion paper. A set of limit states composed of linear functions are used to test the efficiency and possible directional preference of the networks. These networks are shown in the tests and examples to reduce the simulation effort in finite element-based reliability assessment.     

Convergence analysis of hybrid bi-directional associative memory neural networks with discrete delays

In this paper, the dynamical characteristics of hybrid bi-directional associative memory (BAM) neural networks with constant transmission delays are investigated. Without assuming the symmetry of synaptic connection weights and the monotonicity and differentiability of activation functions, the Lyapunov functionals are constructed and Halanay-type inequalities are respectively employed to derive the delay-independent sufficient conditions under which the networks converge exponentially to the equilibria associated with temporally uniform external inputs. Some examples are given to illustrate that the results are less conservative and less restrictive than the previously known results.     

Symptom based diagnostic system for nuclear power plant operations using artificial neural networks

Nuclear power plant experiences a number of transients during its operations. These transients may be due to equipment failure, malfunctioning of process systems and unavailability of safety systems. In such a situation, the plant may result into an abnormal state which is undesired. In case of an undesired plant condition generally known as an initiating event (IE), the operator has to carry out diagnostic and corrective actions. The operator's response may be too late to mitigate or minimize the negative consequences in such scenarios. The objective of this work is to develop an operator support system based on artificial neural networks that will assist the operator to identify the IEs at the earliest stages of their developments. These abnormal plant conditions must be diagnosed and identified through the process instrument readings. A symptom based diagnostic system has been developed to investigate the IEs. The event identification is carried out by using resilient back propagation neural network algorithm. Whenever an event is detected, the system will display the necessary operator actions in addition to the type of IE. The system will also show the graphical trend of relevant parameters. The developed system is able to identify the eight IEs of Narora Atomic Power Station. This paper describes the features of the diagnostic system taking one of the IEs as a case study.