Exploiting intrinsic fluctuations to identify model parameters

Parameterisation of kinetic models plays a central role in computational systems biology. Besides the lack of experimental data of high enough quality, some of the biggest challenges here are identification issues. Model parameters can be structurally non‐identifiable because of functional relationships. Noise in measured data is usually considered to be a nuisance for parameter estimation. However, it turns out that intrinsic fluctuations in particle numbers can make parameters identifiable that were previously non‐identifiable. The authors present a method to identify model parameters that are structurally non‐identifiable in a deterministic framework. The method takes time course recordings of biochemical systems in steady state or transient state as input. Often a functional relationship between parameters presents itself by a one‐dimensional manifold in parameter space containing parameter sets of optimal goodness. Although the system''s behaviour cannot be distinguished on this manifold in a deterministic framework it might be distinguishable in a stochastic modelling framework. Their method exploits this by using an objective function that includes a measure for fluctuations in particle numbers. They show on three example models, immigration‐death, gene expression and Epo‐EpoReceptor interaction, that this resolves the non‐identifiability even in the case of measurement noise with known amplitude. The method is applied to partially observed recordings of biochemical systems with measurement noise. It is simple to implement and it is usually very fast to compute. This optimisation can be realised in a classical or Bayesian fashion.Inspec keywords: biochemistry, physiological models, stochastic processes, measurement errors, fluctuations, parameter estimationOther keywords: model parameter identification, deterministic framework, biochemical system, steady state, transient state, stochastic modelling framework, objective function, immigration‐death model, gene expression, Epo–EpoReceptor interaction, stochastic fluctuations, measurement noise     

Time‐invariant biological networks with feedback loops: structural equation models and structural identifiability

Quantitative analyses of biological networks such as key biological parameter estimation necessarily call for the use of graphical models. While biological networks with feedback loops are common in reality, the development of graphical model methods and tools that are capable of dealing with feedback loops is still in its infancy. Particularly, inadequate attention has been paid to the parameter identifiability problem for biological networks with feedback loops such that unreliable or even misleading parameter estimates may be obtained. In this study, the structural identifiability analysis problem of time‐invariant linear structural equation models (SEMs) with feedback loops is addressed, resulting in a general and efficient solution. The key idea is to combine Mason''s gain with Wright''s path coefficient method to generate identifiability equations, from which identifiability matrices are then derived to examine the structural identifiability of every single unknown parameter. The proposed method does not involve symbolic or expensive numerical computations, and is applicable to a broad range of time‐invariant linear SEMs with or without explicit latent variables, presenting a remarkable breakthrough in terms of generality. Finally, a subnetwork structure of the C. elegans neural network is used to illustrate the application of the authors’ method in practice.Inspec keywords: matrix algebra, least squares approximations, statistical analysis, parameter estimation, biologyOther keywords: structural identifiability analysis problem, time‐invariant linear structural equation models, feedback loops, identifiability equations, time‐invariant linear SEMs, time‐invariant biological networks, graphical model methods, parameter identifiability problem, biological parameter estimation, subnetwork structure, C. elegans neural network     

Estimating parameters of a stochastic cell invasion model with fluorescent cell cycle labelling using approximate Bayesian computation

We develop a parameter estimation method based on approximate Bayesian computation (ABC) for a stochastic cell invasion model using fluorescent cell cycle labelling with proliferation, migration and crowding effects. Previously, inference has been performed on a deterministic version of the model fitted to cell density data, and not all parameters were identifiable. Considering the stochastic model allows us to harness more features of experimental data, including cell trajectories and cell count data, which we show overcomes the parameter identifiability problem. We demonstrate that, while difficult to collect, cell trajectory data can provide more information about the parameters of the cell invasion model. To handle the intractability of the likelihood function of the stochastic model, we use an efficient ABC algorithm based on sequential Monte Carlo. Rcpp and MATLAB implementations of the simulation model and ABC algorithm used in this study are available at https://github.com/michaelcarr-stats/FUCCI.     

Parameterizing state–space models for infectious disease dynamics by generalized profiling: measles in Ontario

Parameter estimation for infectious disease models is important for basic understanding (e.g. to identify major transmission pathways), for forecasting emerging epidemics, and for designing control measures. Differential equation models are often used, but statistical inference for differential equations suffers from numerical challenges and poor agreement between observational data and deterministic models. Accounting for these departures via stochastic model terms requires full specification of the probabilistic dynamics, and computationally demanding estimation methods. Here, we demonstrate the utility of an alternative approach, generalized profiling, which provides robustness to violations of a deterministic model without needing to specify a complete probabilistic model. We introduce novel means for estimating the robustness parameters and for statistical inference in this framework. The methods are applied to a model for pre-vaccination measles incidence in Ontario, and we demonstrate the statistical validity of our inference through extensive simulation. The results confirm that school term versus summer drives seasonality of transmission, but we find no effects of short school breaks and the estimated basic reproductive ratio ℛ₀ greatly exceeds previous estimates. The approach applies naturally to any system for which candidate differential equations are available, and avoids many challenges that have limited Monte Carlo inference for state–space models.     

Bayesian State Space Modeling of Physical Processes in Industrial Hygiene

Abstract

Exposure assessment models are deterministic models derived from physical–chemical laws. In real workplace settings, chemical concentration measurements can be noisy and indirectly measured. In addition, inference on important parameters such as generation and ventilation rates are usually of interest since they are difficult to obtain. In this article, we outline a flexible Bayesian framework for parameter inference and exposure prediction. In particular, we devise Bayesian state space models by discretizing the differential equation models and incorporating information from observed measurements and expert prior knowledge. At each time point, a new measurement is available that contains some noise, so using the physical model and the available measurements, we try to obtain a more accurate state estimate, which can be called filtering. We consider Monte Carlo sampling methods for parameter estimation and inference under nonlinear and non-Gaussian assumptions. The performance of the different methods is studied on computer-simulated and controlled laboratory-generated data. We consider some commonly used exposure models representing different physical hypotheses. Supplementary materials for this article are available online.     

Stochastic stability of quasi non-integrable Hamiltonian systems under parametric excitations of Gaussian and Poisson white noises

The asymptotic Lyapunov stability with probability one of n-degree-of-freedom (n-DOF) quasi non-integrable Hamiltonian systems subject to weakly parametric excitations of combined Gaussian and Poisson white noises is studied by using the largest Lyapunov exponent. First, an n-DOF quasi non-integrable Hamiltonian system subject to weakly parametric excitations of combined Gaussian and Poisson white noises is reduced to a one-dimensional averaged Itô stochastic differential equation (SDE) for Hamiltonian by using the stochastic averaging method for quasi non-integrable Hamiltonian systems. Then, the expression for the Lyapunov exponent of the averaged Itô SDE is derived and the approximately necessary and sufficient condition for the asymptotic Lyapunov stability with probability one of the trivial solution of the original system is obtained. Finally, one example is worked out to illustrate the proposed procedure and its effectiveness is confirmed by comparing with Monte Carlo simulation. It is found that analytical and simulation results agree well.     

Effective implicit finite‐difference method for sensitivity analysis of stiff stochastic discrete biochemical systems

Simulation of cellular processes is achieved through a range of mathematical modelling approaches. Deterministic differential equation models are a commonly used first strategy. However, because many biochemical processes are inherently probabilistic, stochastic models are often called for to capture the random fluctuations observed in these systems. In that context, the Chemical Master Equation (CME) is a widely used stochastic model of biochemical kinetics. Use of these models relies on estimates of kinetic parameters, which are often poorly constrained by experimental observations. Consequently, sensitivity analysis, which quantifies the dependence of systems dynamics on model parameters, is a valuable tool for model analysis and assessment. A number of approaches to sensitivity analysis of biochemical models have been developed. In this study, the authors present a novel method for estimation of sensitivity coefficients for CME models of biochemical reaction systems that span a wide range of time‐scales. They make use of finite‐difference approximations and adaptive implicit tau‐leaping strategies to estimate sensitivities for these stiff models, resulting in significant computational efficiencies in comparison with previously published approaches of similar accuracy, as evidenced by illustrative applications.Inspec keywords: biochemistry, sensitivity analysis, stochastic processes, cellular biophysics, probability, fluctuations, master equation, reaction kinetics, finite difference methodsOther keywords: effective implicit finite‐difference method, sensitivity analysis, stiff stochastic discrete biochemical systems, cellular processes, mathematical modelling, deterministic differential equation models, inherently probabilistic‐stochastic models, random fluctuations, Chemical Master Equation, biochemical kinetics, kinetic parameter estimation, systems dynamics, CME models, biochemical reaction systems, finite‐difference approximations, adaptive implicit tau‐leaping strategies, computational efficiencies     

Continuous analogue to iterative optimization for PDE-constrained inverse problems

The parameters of many physical processes are unknown and have to be inferred from experimental data. The corresponding parameter estimation problem is often solved using iterative methods such as steepest descent methods combined with trust regions. For a few problem classes also continuous analogues of iterative methods are available. In this work, we expand the application of continuous analogues to function spaces and consider PDE (partial differential equation)-constrained optimization problems. We derive a class of continuous analogues, here coupled ODE (ordinary differential equation)–PDE models, and prove their convergence to the optimum under mild assumptions. We establish sufficient bounds for local stability and convergence for the tuning parameter of this class of continuous analogues, the retraction parameter. To evaluate the continuous analogues, we study the parameter estimation for a model of gradient formation in biological tissues. We observe good convergence properties, indicating that the continuous analogues are an interesting alternative to state-of-the-art iterative optimization methods.     

Convergence of Recent Multistep Schemes for a Forward-Backward Stochastic Differential Equation

Convergence analysis is presented for recently proposed multistep schemes, when applied to a special type of forward-backward stochastic differential equations (FBSDEs) that arises in finance and stochastic control. The corresponding $k$-step scheme admits a $k$-order convergence rate in time, when the exact solution of the forward stochastic differential equation (SDE) is given. Our analysis assumes that the terminal conditions and the FBSDE coefficients are sufficiently regular.     

Accuracy of the Michaelis–Menten approximation when analysing effects of molecular noise

Quantitative biology relies on the construction of accurate mathematical models, yet the effectiveness of these models is often predicated on making simplifying approximations that allow for direct comparisons with available experimental data. The Michaelis–Menten (MM) approximation is widely used in both deterministic and discrete stochastic models of intracellular reaction networks, owing to the ubiquity of enzymatic activity in cellular processes and the clear biochemical interpretation of its parameters. However, it is not well understood how the approximation applies to the discrete stochastic case or how it extends to spatially inhomogeneous systems. We study the behaviour of the discrete stochastic MM approximation as a function of system size and show that significant errors can occur for small volumes, in comparison with a corresponding mass-action system. We then explore some consequences of these results for quantitative modelling. One consequence is that fluctuation-induced sensitivity, or stochastic focusing, can become highly exaggerated in models that make use of MM kinetics even if the approximations are excellent in a deterministic model. Another consequence is that spatial stochastic simulations based on the reaction–diffusion master equation can become highly inaccurate if the model contains MM terms.     

Thermo-mechanical buckling and nonlinear free vibration analysis of functionally graded beams on nonlinear elastic foundation

In this paper, thermo-mechanical buckling and nonlinear free vibration analysis of functionally graded (FG) beams on nonlinear elastic foundation are investigated. Nonlinear governing partial differential equation (PDE) of motion is derived based on Euler–Bernoulli assumptions together with Von Karman strain–displacement relation. Based on the Galerkin’s decomposition method, the nonlinear PDE governing equation is reduced to a nonlinear ordinary differential equation (ODE). He’s variational method is employed to obtain a simple and efficient approximate closed form solution for the resulted nonlinear ODE. Comparison between results of the present work and those available in literature shows accuracy of the presented expressions. Some new results for the thermo-mechanical buckling and nonlinear free vibration analysis of the FG beams such as the effects of vibration amplitude, material inhomogeneity, nonlinear elastic foundation, boundary conditions, geometric parameter and thermal loading are presented to be used in future references.     

Modulation of stochastic diffusion by wave motion

Pollutants that are chemically inert flow with the carrier fluid passively while diffuse at the same time. In this study, the stochastic diffusion behavior of the passive pollutant in a progressive or standing wave field is examined with analytical means. Our focus is on the nonlinear interactions between the stochastic diffusion and the deterministic wave motions, and we limit the scope to cases whereby a small parameter, ε, exists between the advective and diffusive displacements, which then allows a perturbation analysis to be performed. With a sinusoidal progressive wave, the results show that the deterministic wave motion can either increase or decrease the embedded stochastic diffusion depending on the wave characteristics. Longer wave lengths and shorter wave periods tend to promote diffusion significantly, while shorter wave lengths and longer wave periods act in the opposite manner but with a much smaller effect. An analysis of the standing wave motion, represented by a combination of left and right moving progressive waves, shows that the effects due to two opposing waves to the stochastic diffusion can be superimposed.     

Coordination of gene expression noise with cell size: analytical results for agent-based models of growing cell populations

The chemical master equation and the Gillespie algorithm are widely used to model the reaction kinetics inside living cells. It is thereby assumed that cell growth and division can be modelled through effective dilution reactions and extrinsic noise sources. We here re-examine these paradigms through developing an analytical agent-based framework of growing and dividing cells accompanied by an exact simulation algorithm, which allows us to quantify the dynamics of virtually any intracellular reaction network affected by stochastic cell size control and division noise. We find that the solution of the chemical master equation—including static extrinsic noise—exactly agrees with the agent-based formulation when the network under study exhibits stochastic concentration homeostasis, a novel condition that generalizes concentration homeostasis in deterministic systems to higher order moments and distributions. We illustrate stochastic concentration homeostasis for a range of common gene expression networks. When this condition is not met, we demonstrate by extending the linear noise approximation to agent-based models that the dependence of gene expression noise on cell size can qualitatively deviate from the chemical master equation. Surprisingly, the total noise of the agent-based approach can still be well approximated by extrinsic noise models.     

Autonomous Boolean modelling of developmental gene regulatory networks

During early embryonic development, a network of regulatory interactions among genes dynamically determines a pattern of differentiated tissues. We show that important timing information associated with the interactions can be faithfully represented in autonomous Boolean models in which binary variables representing expression levels are updated in continuous time, and that such models can provide a direct insight into features that are difficult to extract from ordinary differential equation (ODE) models. As an application, we model the experimentally well-studied network controlling fly body segmentation. The Boolean model successfully generates the patterns formed in normal and genetically perturbed fly embryos, permits the derivation of constraints on the time delay parameters, clarifies the logic associated with different ODE parameter sets and provides a platform for studying connectivity and robustness in parameter space. By elucidating the role of regulatory time delays in pattern formation, the results suggest new types of experimental measurements in early embryonic development.     

Response of nonlinear oscillators with fractional derivative elements under evolutionary stochastic excitations: A Path Integral approach based on Laplace’s method of integration

In this paper, an approximate analytical technique is developed for determining the non-stationary response amplitude probability density function (PDF) of nonlinear/hysteretic oscillators endowed with fractional element and subjected to evolutionary excitations. This is achieved by a novel formulation of the Path Integral (PI) approach. Specifically, a stochastic averaging/linearization treatment of the original fractional order governing equation of motion yields a first-order stochastic differential equation (SDE) for the oscillator response amplitude. Associated with this first-order SDE is the Chapman–Kolmogorov (CK) equation governing the evolution in time of the non-stationary response amplitude PDF. Next, the PI technique is employed, which is based on a discretized version of the CK equation solved in short time steps. This is done relying on the Laplace’s method of integration which yields an approximate analytical solution of the integral involved in the CK equation. In this manner, the repetitive integrations generally required in the classical numerical implementation of the procedure are avoided. Thus, the non-stationary response amplitude PDF is approximately determined in closed-form in a computationally efficient manner. Notably, the technique can also account for arbitrary excitation evolutionary power spectrum forms, even of the non-separable kind. Applications to oscillators with Van der Pol and Duffing type nonlinear restoring force models, and Preisach hysteretic models, are presented. Appropriate comparisons with Monte Carlo simulation data are shown, demonstrating the efficiency and accuracy of the proposed approach.     

Learning differential equation models from stochastic agent-based model simulations

Agent-based models provide a flexible framework that is frequently used for modelling many biological systems, including cell migration, molecular dynamics, ecology and epidemiology. Analysis of the model dynamics can be challenging due to their inherent stochasticity and heavy computational requirements. Common approaches to the analysis of agent-based models include extensive Monte Carlo simulation of the model or the derivation of coarse-grained differential equation models to predict the expected or averaged output from the agent-based model. Both of these approaches have limitations, however, as extensive computation of complex agent-based models may be infeasible, and coarse-grained differential equation models can fail to accurately describe model dynamics in certain parameter regimes. We propose that methods from the equation learning field provide a promising, novel and unifying approach for agent-based model analysis. Equation learning is a recent field of research from data science that aims to infer differential equation models directly from data. We use this tutorial to review how methods from equation learning can be used to learn differential equation models from agent-based model simulations. We demonstrate that this framework is easy to use, requires few model simulations, and accurately predicts model dynamics in parameter regions where coarse-grained differential equation models fail to do so. We highlight these advantages through several case studies involving two agent-based models that are broadly applicable to biological phenomena: a birth–death–migration model commonly used to explore cell biology experiments and a susceptible–infected–recovered model of infectious disease spread.     

16MnR钢疲劳可靠性分析单随机变量模型

本文提出了采用一个随机变量描述疲劳裂纹扩展统计不确定性方法,并且通过极大似然方法和二阶矩方法进行疲劳可靠性分析。通过21个紧凑拉伸试件的疲劳试验,表明本文提出的方法满足工程中疲劳可靠性评估精度的要求。     

Ion channel model reduction using manifold boundaries

Mathematical models of voltage-gated ion channels are used in basic research, industrial and clinical settings. These models range in complexity, but typically contain numerous variables representing the proportion of channels in a given state, and parameters describing the voltage-dependent rates of transition between states. An open problem is selecting the appropriate degree of complexity and structure for an ion channel model given data availability. Here, we simplify a model of the cardiac human Ether-à-go-go related gene (hERG) potassium ion channel, which carries cardiac I_Kr, using the manifold boundary approximation method (MBAM). The MBAM approximates high-dimensional model-output manifolds by reduced models describing their boundaries, resulting in models with fewer parameters (and often variables). We produced a series of models of reducing complexity starting from an established five-state hERG model with 15 parameters. Models with up to three fewer states and eight fewer parameters were shown to retain much of the predictive capability of the full model and were validated using experimental hERG1a data collected in HEK293 cells at 37°C. The method provides a way to simplify complex models of ion channels that improves parameter identifiability and will aid in future model development.     

Application of the generalized perturbation-based stochastic boundary element method to the elastostatics

This paper is entirely devoted to the demonstration of a solution for some boundary value problems of isotropic linear elastostatics with random parameters using the boundary element method. The stochastic perturbation technique in its general nth-order Taylor series expansion version is used to express all the random parameters and the state functions of the problem. These expansions inserted in the classical deterministic equilibrium statement return up to the nth-order (both PDEs and matrix) equations. Contrary to the previous implementations of the stochastic perturbation technique, any order partial derivatives with respect to the random input are derived from the deterministic structural response function (SRF) at a given point. This function is approximated using polynomials by the least-squares method from the multiple solution of the initial deterministic problem solved for the expectations of random structural parameters. First two probabilistic moments have been computed symbolically here using the computational MAPLE environment, also as the polynomial expressions including perturbation parameter ε. It should be mentioned that such a generalized perturbation approach makes it possible to analyze all types of random variables (not only Gaussian) and to compute even higher probabilistic moments with a priori given accuracy. The entire methodology can be implemented after minor modifications to analyze nonlinear phenomena for both statics and dynamics of even heterogeneous domains.