Tensor methods for parameter estimation and bifurcation analysis of stochastic reaction networks

Stochastic modelling of gene regulatory networks provides an indispensable tool for understanding how random events at the molecular level influence cellular functions. A common challenge of stochastic models is to calibrate a large number of model parameters against the experimental data. Another difficulty is to study how the behaviour of a stochastic model depends on its parameters, i.e. whether a change in model parameters can lead to a significant qualitative change in model behaviour (bifurcation). In this paper, tensor-structured parametric analysis (TPA) is developed to address these computational challenges. It is based on recently proposed low-parametric tensor-structured representations of classical matrices and vectors. This approach enables simultaneous computation of the model properties for all parameter values within a parameter space. The TPA is illustrated by studying the parameter estimation, robustness, sensitivity and bifurcation structure in stochastic models of biochemical networks. A Matlab implementation of the TPA is available at http://www.stobifan.org.     

Quasi-robust control of biochemical reaction networks via stochastic morphing

One of the main objectives of synthetic biology is the development of molecular controllers that can manipulate the dynamics of a given biochemical network that is at most partially known. When integrated into smaller compartments, such as living or synthetic cells, controllers have to be calibrated to factor in the intrinsic noise. In this context, biochemical controllers put forward in the literature have focused on manipulating the mean (first moment) and reducing the variance (second moment) of the target molecular species. However, many critical biochemical processes are realized via higher-order moments, particularly the number and configuration of the probability distribution modes (maxima). To bridge the gap, we put forward the stochastic morpher controller that can, under suitable timescale separations, morph the probability distribution of the target molecular species into a predefined form. The morphing can be performed at a lower-resolution, allowing one to achieve desired multi-modality/multi-stability, and at a higher-resolution, allowing one to achieve arbitrary probability distributions. Properties of the controller, such as robustness and convergence, are rigorously established, and demonstrated on various examples. Also proposed is a blueprint for an experimental implementation of stochastic morpher.     

An efficient method for estimating global sensitivity indices

An efficient method is proposed to estimate the first‐order global sensitivity indices based on failure probability and variance by using maximum entropy theory and Nataf transformation. The computational cost of this proposed method is quite small, and the proposed method can efficiently overcome the ‘dimensional curse’ due to dimensional reduction technique. Ideas for the estimation of higher‐order sensitivity indices are discussed. Copyright © 2016 John Wiley & Sons, Ltd.     

结构时变可靠度计算的全随机过程模型

为了对既有结构的可靠性进行正确评估,必须考虑时间变化的影响,对目前可靠度分析中采用的只考虑荷载随时间变化的半随机过程模型进行改进。基于结构抗力和作用效应相互独立的基本假设,充分依据作用效应和抗力的时变特性,考虑结构抗力为独立增量过程,并计算了抗力的自相关系数,得到了计算结构失效概率的近似算法,建立了结构时变可靠度计算的全随机过程模型。通过实例验证,该方法简单易行,便于工程应用,为基于时变可靠度理论的既有结构评估和寿命预测提供理论依据。     

On reaction network implementations of neural networks

This paper is concerned with the utilization of deterministically modelled chemical reaction networks for the implementation of (feed-forward) neural networks. We develop a general mathematical framework and prove that the ordinary differential equations (ODEs) associated with certain reaction network implementations of neural networks have desirable properties including (i) existence of unique positive fixed points that are smooth in the parameters of the model (necessary for gradient descent) and (ii) fast convergence to the fixed point regardless of initial condition (necessary for efficient implementation). We do so by first making a connection between neural networks and fixed points for systems of ODEs, and then by constructing reaction networks with the correct associated set of ODEs. We demonstrate the theory by constructing a reaction network that implements a neural network with a smoothed ReLU activation function, though we also demonstrate how to generalize the construction to allow for other activation functions (each with the desirable properties listed previously). As there are multiple types of ‘networks’ used in this paper, we also give a careful introduction to both reaction networks and neural networks, in order to disambiguate the overlapping vocabulary in the two settings and to clearly highlight the role of each network’s properties.     

Sparse grids‐based stochastic approximations with applications to aerodynamics sensitivity analysis

This work compares sample‐based polynomial surrogates, well suited for moderately high‐dimensional stochastic problems. In particular, generalized polynomial chaos in its sparse pseudospectral form and stochastic collocation methods based on both isotropic and dimension‐adapted sparse grids are considered. Both classes of approximations are compared, and an improved version of a stochastic collocation with dimension adaptivity driven by global sensitivity analysis is proposed. The stochastic approximations efficiency is assessed on multivariate test function and airfoil aerodynamics simulations. The latter study addresses the probabilistic characterization of global aerodynamic coefficients derived from viscous subsonic steady flow about a NACA0015 airfoil in the presence of geometrical and operational uncertainties with both simplified aerodynamics model and Reynolds‐Averaged Navier‐Stokes (RANS) simulation. Sparse pseudospectral and collocation approximations exhibit similar level of performance for isotropic sparse simulation ensembles. Computational savings and accuracy gain of the proposed adaptive stochastic collocation driven by Sobol' indices are patent but remain problem‐dependent. Copyright © 2015 John Wiley & Sons, Ltd.     

An optimization algorithm based on stochastic sensitivity analysis for noisy objective landscapes

A function minimization algorithm that updates solutions based on approximated derivative information is proposed. The algorithm generates sample points with Gaussian white noise, and approximates derivatives based on stochastic sensitivity analysis. Unlike standard trust region methods which calculate gradients with n or more sample points, where n is the number of variables, the proposed algorithm allows the number of sample points M to be less than n. Furthermore, it ignores small amounts of noise within a trust region. This paper addresses the following two questions: how does the derivative approximation become worse when the number of sample points is small? Can the algorithm find good solutions with inexact derivative information when the objective landscape is noisy? Through intensive numerical experiments using quadratic functions, the algorithm is shown to be able to approximate derivatives when M is about n/10 or more. The experiments using a formulation of the traveling salesman problem show that the algorithm can find reasonably good solutions for noisy objective landscapes with inexact derivatives information.     

Redundancy analysis for repairable multi-state system by using combined stochastic processes methods and universal generating function technique

This paper discusses a type of redundancy that is typical in a multi-state system. It considers two interconnected multi-state systems where one multi-state system can satisfy its own stochastic demand and also can provide abundant resource (performance) to another system in order to improve the assisted system reliability. Traditional methods are usually not effective enough for reliability analysis for such multi-state systems because of the “dimensional curse” problem. This paper presents a new method for reliability evaluation for the repairable multi-state system considering such kind of redundancy. The proposed method is based on the combination of the universal generating function technique and random processes methods. The numerical example is presented to illustrate the proposed method.     

An efficient method for reliability evaluation of multistate networks given all minimal path vectors

The multistate networks under consideration consist of a source node, a sink node, and some independent failure-prone components in between the nodes. The components can work at different levels of capacity. For such a network, we are interested in evaluating the probability that the flow from the source node to the sink node is equal to or greater than a demanded flow of d units. A general method for reliability evaluation of such multistate networks is using minimal path (cut) vectors. A minimal path vector to system state d is called a d-MP. Approaches for generating all d-MPs have been reported. Given that all d-MPs have been found, the issue becomes how to evaluate the probability of the union of the events that the component state vector is greater than or equal to at least one of the d-MPs. There is a need for a more efficient method of determining the probability of this union of events. In this paper, we report an efficient recursive algorithm for this union probability evaluation based on the Sum of Disjoint Products (SDP) principle, and name it the Recursive Sum of Disjoint Products (RSDP) algorithm. The basic idea is that, based on the SDP principle and a specially defined “maximum” operator, “⊕”, the probability of a union with L vectors can be calculated via calculating the probabilities of several unions with L-1 vectors or less. The correctness of RSDP is illustrated. The efficiency of this algorithm is investigated by comparing it with an existing algorithm that is generally accepted to be efficient. It is found that RSDP is more efficient than the existing algorithm when the number of components of a system is not too small. RSDP provides us with an efficient, systematic and simple approach for evaluating multistate network reliability given all d-MPs.     

Stochastic analysis of biochemical reaction networks with absolute concentration robustness

It has recently been shown that structural conditions on the reaction network, rather than a ‘fine-tuning’ of system parameters, often suffice to impart ‘absolute concentration robustness’ (ACR) on a wide class of biologically relevant, deterministically modelled mass-action systems. We show here that fundamentally different conclusions about the long-term behaviour of such systems are reached if the systems are instead modelled with stochastic dynamics and a discrete state space. Specifically, we characterize a large class of models that exhibit convergence to a positive robust equilibrium in the deterministic setting, whereas trajectories of the corresponding stochastic models are necessarily absorbed by a set of states that reside on the boundary of the state space, i.e. the system undergoes an extinction event. If the time to extinction is large relative to the relevant timescales of the system, the process will appear to settle down to a stationary distribution long before the inevitable extinction will occur. This quasi-stationary distribution is considered for two systems taken from the literature, and results consistent with ACR are recovered by showing that the quasi-stationary distribution of the robust species approaches a Poisson distribution.     

基于随机学习的异构蜂窝网络离散功率控制机制

针对宏基站和低功率小基站重叠覆盖共享频谱的异构网络中干扰抑制问题,提出了一种基于随机学习理论的基站下行离散功率控制机制,以便在保证宏基站传输的最低传输质量要求下,最大化网络满足传输质量要求的小基站传输个数。该方法将每个小基站作为随机学习自动机,维持一个概率向量用于传输功率选择。算法迭代过程中,小基站独立地基于自身选择的传输功率以及网络反馈的信息,即满足传输质量要求的小基站传输的个数,来更新概率向量,调整传输功率,直到系统达到均衡状态。仿真结果表明,基于随机学习的离散功率控制方法具有很好的收敛性和寻优性能,能有效提高系统可支持的满足传输质量要求的小基站传输数量,适用实际系统。     

A heuristic method for obtaining quasi ARL‐unbiased p‐Charts

It is known that control charts based on equal tail probability limits are ARL biased when the distribution of the plotted statistic is skewed. This is the case for p‐Charts that serve to monitor processes on the basis of the binomial distribution. For the particular case of the standard three‐sigma Shewhart p‐Chart, which is built on the basis of the binomial to normal distribution approximation, this ARL‐biased condition is particularly severe, and it greatly affects its monitoring capability. Surprisingly, in spite of this, the standard p‐Chart is still widely used and taught. Through a literature search, it was identified that several, simple to use, improved alternative p‐Charts had been proposed over the years; however, at first instance, it was not possible to determine which of them was the best. In order to identify the alternative that excelled, an ARL performance comparison was carried out in terms of their ARL bias severity level (ARL_BSL) and their In‐Control ARL (ARL₀). The results showed that even the best performing alternative charts would often be ARL‐biased or have nonoptimal ARL₀. To improve on the existing alternatives, the “Kmod p‐Chart” was developed; it offers easiness of use, superior ARL performance, and a simple and effective method for verifying its ARL‐bias condition.     

An implicit method for the analysis of transient flows in pipe networks

Existing methods for the analysis of transient flows in pipe networks are often geared towards certain types of flows such as gas flows vis‐à‐vis liquid flows or isothermal flows vis‐à‐vis non‐isothermal flows. Also, simplifying assumptions are often made which introduce inaccuracies when the method is applied outside the domain for which it was originally intended. This paper describes an implicit finite difference method based on the simultaneous pressure correction approach which is valid for both liquid and gas flows, for both isothermal and non‐isothermal flows and for both fast and slow transients. The problematic convective acceleration term in the momentum equation, often neglected in other methods, is retained but eliminated by casting the momentum equation in an alternative form. The accuracy and stability of the method, depending on a time‐step weighing factor α, are illustrated by analysing fast transients in a pipeline and simple branching network. The proposed method compares very well with the second‐order‐method of characteristics and the two‐step Lax–Wendroff method. The advantages of the present method is its speed over a range of problems including both fast and slow transients, its accuracy, its stability and its flexibility. Copyright © 2001 John Wiley & Sons, Ltd.     

Highly efficient general method for sensitivity analysis of eigenvectors with repeated eigenvalues without passing through adjacent eigenvectors

It is well known that the sensitivity analysis of the eigenvectors corresponding to multiple eigenvalues is a difficult problem. The main difficulty is that for given multiple eigenvalues, the eigenvector derivatives can be computed for a specific eigenvector basis, the so-called adjacent eigenvector basis. These adjacent eigenvectors depend on individual variables, which makes the eigenvector derivative calculation elaborate and expensive from a computational perspective. This research presents a method that avoids passing through adjacent eigenvectors in the calculation of the partial derivatives of any prescribed eigenvector basis. As our method fits into the adjoint sensitivity analysis , it is efficient for computing the complete Jacobian matrix because the adjoint variables are independent of each variable. Thus our method clarifies and unifies existing theories on eigenvector sensitivity analysis. Moreover, it provides a highly efficient computational method with a significant saving of the computational cost. Additional benefits of our approach are that one does not have to solve a deficient linear system and that the method is independent of the existence of repeated eigenvalue derivatives of the multiple eigenvalues. Our method covers the case of eigenvectors associated to a single eigenvalue. Some examples are provided to validate the present approach.     

On the optimal design of radial basis function neural networks for the analysis of nonlinear stochastic systems

In this paper, an iterative selection strategy of Gaussian neurons for radial basis function neural networks (RBFNN) is proposed when the RBFNN method is applied to obtain the steady-state solution of the Fokker–Planck–Kolmogorov (FPK) equation. A performance index is introduced to rank neurons. Top rank neurons are selected, leading to a RBFNN with optimal number and locations of Gaussian neurons for the FPK equation under consideration. The statistical properties of the performance index are studied. It is found that the index assigned to the $j$th neuron is proportional to the probability of the system falling into the small neighborhood of the mean of this neuron as well as proportional to the weight of the neuron. The RBFNN method with the optimally selected neurons is then applied to several challenging examples of nonlinear stochastic systems in 2, 3 and 4 dimensional state space. The RBFNN solutions are also compared with the results of extensive Monte Carlo simulations. It is observed that the RBFNN method with optimally selected neurons by the proposed iterative algorithm is much more efficient than the RBFNN method with uniformly distributed neurons, and is very accurate in terms of the root mean squared (RMS) errors of the FPK equation or the RMS errors of the PDF solution compared with simulation results.     

Information-theoretic sensitivity analysis: a general method for credit assignment in complex networks.

Most systems can be represented as networks that couple a series of nodes to each other via one or more edges, with typically unknown equations governing their quantitative behaviour. A major question then pertains to the importance of each of the elements that act as system inputs in determining the output(s). We show that any such system can be treated as a 'communication channel' for which the associations between inputs and outputs can be quantified via a decomposition of their mutual information into different components characterizing the main effect of individual inputs and their interactions. Unlike variance-based approaches, our novel methodology can easily accommodate correlated inputs.     

Theoretical development of stochastic delay analysis and forecast method

Abstract

Schedule delay analysis and claim management for construction projects have received much attention recently due to an increasing number of construction disputes. Most of the available delay analysis methods employ a deterministic approach that uses deterministic activity duration in as-planned schedules for schedule comparison. However, the duration of construction projects is naturally uncertain. Based on a mathematical-model delay analysis method, this study proposes an innovative approach, the Stochastic Delay Analysis and Forecast method (the SDAF method), which combines probabilistic activity duration and systematic delay analysis procedures to predict the contribution of a single activity’s delay and its probability to cause overall project delay. This paper proposes a methodology and examines its results by means of a simulated case. Although this study has simplified practical problems, preliminary study results are valuable and form a basis for further research. Namely, these study results provide not only a stochastic delay analysis and forecast method but also a fundamental basis for developing advanced schedule delay analysis methodology to help in solving complicated schedule delay problems in construction projects.     

多源随机激励系统的参数灵敏度分析

采用虚拟激励法,多次利用状态空间解耦、Kronecker代数及矩阵微分理论等,推导出多源随机激励下响应均方根的参数灵敏度表达式,并将其应用于汽车平顺性分析。根据理论计算的灵敏度大小及工程实际要求,选取优化变量,对轻型货车整车模型进行了平顺性优化     

An efficient method to estimate sorption isotherm curve coefficients

This paper aims at estimating the sorption isotherm coefficients of a wood fiber material using experimental data. First, the mathematical model, based on convective transport of moisture, the Optimal Experiment Design (OED) and the experimental set-up are presented. Then, measurements of relative humidity within the material are carried out, after searching the OED using the computation of the sensitivity functions and a priori values of the unknown parameters. It enables to plan the experimental conditions in terms of sensor positioning and boundary conditions out of 20 possible designs, ensuring the best accuracy for the identification method and, thus, for the estimated parameter. After the measurements, the parameter estimation problem is solved. The determined sorption isotherm coefficients calibrate the numerical model to fit better the experimental data. However, some discrepancies still appear since the hysteresis effects on the sorption capacity are not included in the model. Therefore, the latter is improved proposing an additional differential equation for the sorption capacity to consider the hysteresis effects. The OED approach is developed for the estimation of five of the coefficients involved in the hysteresis model. To conclude, the prediction of the model with hysteresis have better reliability when compared to the experimental observations.