Learning differential equation models from stochastic agent-based model simulations

Agent-based models provide a flexible framework that is frequently used for modelling many biological systems, including cell migration, molecular dynamics, ecology and epidemiology. Analysis of the model dynamics can be challenging due to their inherent stochasticity and heavy computational requirements. Common approaches to the analysis of agent-based models include extensive Monte Carlo simulation of the model or the derivation of coarse-grained differential equation models to predict the expected or averaged output from the agent-based model. Both of these approaches have limitations, however, as extensive computation of complex agent-based models may be infeasible, and coarse-grained differential equation models can fail to accurately describe model dynamics in certain parameter regimes. We propose that methods from the equation learning field provide a promising, novel and unifying approach for agent-based model analysis. Equation learning is a recent field of research from data science that aims to infer differential equation models directly from data. We use this tutorial to review how methods from equation learning can be used to learn differential equation models from agent-based model simulations. We demonstrate that this framework is easy to use, requires few model simulations, and accurately predicts model dynamics in parameter regions where coarse-grained differential equation models fail to do so. We highlight these advantages through several case studies involving two agent-based models that are broadly applicable to biological phenomena: a birth–death–migration model commonly used to explore cell biology experiments and a susceptible–infected–recovered model of infectious disease spread.     

Parameterizing state–space models for infectious disease dynamics by generalized profiling: measles in Ontario

Parameter estimation for infectious disease models is important for basic understanding (e.g. to identify major transmission pathways), for forecasting emerging epidemics, and for designing control measures. Differential equation models are often used, but statistical inference for differential equations suffers from numerical challenges and poor agreement between observational data and deterministic models. Accounting for these departures via stochastic model terms requires full specification of the probabilistic dynamics, and computationally demanding estimation methods. Here, we demonstrate the utility of an alternative approach, generalized profiling, which provides robustness to violations of a deterministic model without needing to specify a complete probabilistic model. We introduce novel means for estimating the robustness parameters and for statistical inference in this framework. The methods are applied to a model for pre-vaccination measles incidence in Ontario, and we demonstrate the statistical validity of our inference through extensive simulation. The results confirm that school term versus summer drives seasonality of transmission, but we find no effects of short school breaks and the estimated basic reproductive ratio ℛ₀ greatly exceeds previous estimates. The approach applies naturally to any system for which candidate differential equations are available, and avoids many challenges that have limited Monte Carlo inference for state–space models.     

New model diagnostics for spatio-temporal systems in epidemiology and ecology

A cardinal challenge in epidemiological and ecological modelling is to develop effective and easily deployed tools for model assessment. The availability of such methods would greatly improve understanding, prediction and management of disease and ecosystems. Conventional Bayesian model assessment tools such as Bayes factors and the deviance information criterion (DIC) are natural candidates but suffer from important limitations because of their sensitivity and complexity. Posterior predictive checks, which use summary statistics of the observed process simulated from competing models, can provide a measure of model fit but appropriate statistics can be difficult to identify. Here, we develop a novel approach for diagnosing mis-specifications of a general spatio-temporal transmission model by embedding classical ideas within a Bayesian analysis. Specifically, by proposing suitably designed non-centred parametrization schemes, we construct latent residuals whose sampling properties are known given the model specification and which can be used to measure overall fit and to elicit evidence of the nature of mis-specifications of spatial and temporal processes included in the model. This model assessment approach can readily be implemented as an addendum to standard estimation algorithms for sampling from the posterior distributions, for example Markov chain Monte Carlo. The proposed methodology is first tested using simulated data and subsequently applied to data describing the spread of Heracleum mantegazzianum (giant hogweed) across Great Britain over a 30-year period. The proposed methods are compared with alternative techniques including posterior predictive checking and the DIC. Results show that the proposed diagnostic tools are effective in assessing competing stochastic spatio-temporal transmission models and may offer improvements in power to detect model mis-specifications. Moreover, the latent-residual framework introduced here extends readily to a broad range of ecological and epidemiological models.     

Deterministic inference for stochastic systems using multiple shooting and a linear noise approximation for the transition probabilities

Estimating model parameters from experimental data is a crucial technique for working with computational models in systems biology. Since stochastic models are increasingly important, parameter estimation methods for stochastic modelling are also of increasing interest. This study presents an extension to the ‘multiple shooting for stochastic systems (MSS)’ method for parameter estimation. The transition probabilities of the likelihood function are approximated with normal distributions. Means and variances are calculated with a linear noise approximation on the interval between succeeding measurements. The fact that the system is only approximated on intervals which are short in comparison with the total observation horizon allows to deal with effects of the intrinsic stochasticity. The study presents scenarios in which the extension is essential for successfully estimating the parameters and scenarios in which the extension is of modest benefit. Furthermore, it compares the estimation results with reversible jump techniques showing that the approximation does not lead to a loss of accuracy. Since the method is not based on stochastic simulations or approximative sampling of distributions, its computational speed is comparable with conventional least‐squares parameter estimation methods.Inspec keywords: stochastic systems, parameter estimation, probability, least squares approximationsOther keywords: deterministic inference, stochastic systems, multiple shooting, linear noise approximation, transition probabilities, systems biology, parameter estimation methods, likelihood function, normal distributions, intrinsic stochasticity effects, reversible jump techniques, approximative sampling, conventional least‐squares parameter estimation methods     

磁流变阻尼器的模糊逼近

由于磁流变液具有非线性特性，所以磁流变阻尼器的输入输出问具有很强的非线性关系。可准确描述其非线性特性的磁流变阻尼器正模型通常非常复杂，难以直接得到逆模型。考虑到某些模糊系统的万能逼近能力，本文提出用模糊系统来逼近磁流变阻尼器逆模型的新思路。根据自适应神经模糊推理系统原理，设计两个模糊系统分别逼近磁流变阻尼器的正模型和逆模型。研究结果表明：无论是正模型还是逆模型。对于训练数据，模糊系统均可以准确逼近，而对于检验数据也可比较准确逼近。正模型的逼近效果稍好，若要提高逆模型ANFIS的逼近精度．将以增加系统复杂性为代价。模糊逼近可以推广到其它的磁流变阻尼器模型中，特别是可对正模型未知的磁流变阻尼器进行建模与控制。     

Bayesian reliability: Combining information

ABSTRACT

One of the most powerful features of Bayesian analyses is the ability to combine multiple sources of information in a principled way to perform inference. This feature can be particularly valuable in assessing the reliability of systems where testing is limited. At their most basic, Bayesian methods for reliability develop informative prior distributions using expert judgment or similar systems. Appropriate models allow the incorporation of many other sources of information, including historical data, information from similar systems, and computer models. We introduce the Bayesian approach to reliability using several examples and point to open problems and areas for future work.     

Improved eigenfrequencies of mechanical systems by combining complementary models

In this paper, we propose a novel approach to combine two complementary numerical models of the dynamical behaviour of mechanical systems: primal (compatible) and dual (equilibrated). This combined model provides an improved estimate of the eigenfrequencies of the system by feeding each model with information from the other. Numerical solutions obtained from both the fundamental models and from the proposed approach are presented and studied. This approach will be applicable to any eigenvalue problem associated with a PDE that can be expressed by complementary numerical models.     

A Bayesian generalized Eyring-Weibull accelerated life testing model

In this paper, a novel approach to a Bayesian accelerated life testing model is presented. The Weibull distribution is used as the life distribution and the generalized Eyring model as the time transformation function. This is a model that allows for the use of more than one stressor, whereas other commonly used acceleration models, such as the Arrhenius and power law models, incorporate one stressor. The use of the generalized Eyring-Weibull model developed in this paper is demonstrated in a case study, where Markov chain Monte Carlo methods are utilized to generate samples for posterior inference.     

Transmission network of the 2014–2015 Ebola epidemic in Sierra Leone

Understanding the growth and spatial expansion of (re)emerging infectious disease outbreaks, such as Ebola and avian influenza, is critical for the effective planning of control measures; however, such efforts are often compromised by data insufficiencies and observational errors. Here, we develop a spatial–temporal inference methodology using a modified network model in conjunction with the ensemble adjustment Kalman filter, a Bayesian inference method equipped to handle observational errors. The combined method is capable of revealing the spatial–temporal progression of infectious disease, while requiring only limited, readily compiled data. We use this method to reconstruct the transmission network of the 2014–2015 Ebola epidemic in Sierra Leone and identify source and sink regions. Our inference suggests that, in Sierra Leone, transmission within the network introduced Ebola to neighbouring districts and initiated self-sustaining local epidemics; two of the more populous and connected districts, Kenema and Port Loko, facilitated two independent transmission pathways. Epidemic intensity differed by district, was highly correlated with population size (r = 0.76, p = 0.0015) and a critical window of opportunity for containing local Ebola epidemics at the source (ca one month) existed. This novel methodology can be used to help identify and contain the spatial expansion of future (re)emerging infectious disease outbreaks.     

A Decomposition Strategy for the Variational Inference of Complex Systems

Markov chain Monte Carlo approaches have been widely used for Bayesian inference. The drawback of these methods is that they can be computationally prohibitive especially when complex models are analyzed. In such cases, variational methods may provide an efficient and attractive alternative. However, the variational methods reported to date are applicable to relatively simple models and most are based on a factorized approximation to the posterior distribution. Here, we propose a variational approach that is capable of handling models that consist of a system of differential-algebraic equations and whose posterior approximation can be represented by a multivariate distribution. Under the proposed approach, the solution of the variational inference problem is decomposed into three steps: a maximum a posteriori optimization, which is facilitated by using an orthogonal collocation approach, a preprocessing step, which is based on the estimation of the eigenvectors of the posterior covariance matrix, and an expected propagation optimization problem. To tackle multivariate integration, we employ quadratures derived from the Smolyak rule (sparse grids). Examples are reported to elucidate the advantages and limitations of the proposed methodology. The results are compared to the solutions obtained from a Markov chain Monte Carlo approach. It is demonstrated that significant computational savings can be gained using the proposed approach. This article has supplementary material online.     

分数指数模型的热力学分析及其应用

本文论证了两种经典粘弹性固体模型的等价性并指出了其存在的问题。给出了热力学对分数指数模型 [1]参数的限制条件。计算与实验结果比较表明:因为该模型具有适当多的参数,采用同一组参数可以做到同时与同一材料的蠕变和松弛试验结果很好吻合;并能做到松弛模量和蠕变柔量的Stieltjes卷积近似等于单位阶跃函数;在很宽广的频率范围内能同时很好地模拟真实材料的存储模量和损耗模量。由于其计算速度快,能与大多数真实材料的性能实验结果相拟合,可以广泛应用于工程实际中的粘弹性静力和动力问题的计算。     

Insights into mucosal innate responses to Escherichia coli O157 : H7 colonization of cattle by mathematical modelling of excretion dynamics

Mathematical model-based statistical inference applied to within-host dynamics of infectious diseases can help dissect complex interactions between hosts and microbes. This work has applied advances in model-based inference to understand colonization of cattle by enterohaemorrhagic Escherichia coli O157 : H7 at the terminal rectum. A mathematical model was developed based on niche replication and transition rates at this site. A nested-model comparison, applied to excretion curves from 25 calves, was used to reduce complexity while maintaining integrity. We conclude that, 5–9 days post inoculation, the innate immune response negates bacterial replication on the epithelium and either reduces attachment to or increases detachment from the epithelium of the terminal rectum. Thus, we provide a broadly applicable model that gives novel insights into bacterial replication rates in vivo and the timing and impact of host responses.     

Crash risk analysis for Shanghai urban expressways: A Bayesian semi-parametric modeling approach

Urban expressway systems have been developed rapidly in recent years in China; it has become one key part of the city roadway networks as carrying large traffic volume and providing high traveling speed. Along with the increase of traffic volume, traffic safety has become a major issue for Chinese urban expressways due to the frequent crash occurrence and the non-recurrent congestions caused by them. For the purpose of unveiling crash occurrence mechanisms and further developing Active Traffic Management (ATM) control strategies to improve traffic safety, this study developed disaggregate crash risk analysis models with loop detector traffic data and historical crash data. Bayesian random effects logistic regression models were utilized as it can account for the unobserved heterogeneity among crashes. However, previous crash risk analysis studies formulated random effects distributions in a parametric approach, which assigned them to follow normal distributions. Due to the limited information known about random effects distributions, subjective parametric setting may be incorrect. In order to construct more flexible and robust random effects to capture the unobserved heterogeneity, Bayesian semi-parametric inference technique was introduced to crash risk analysis in this study. Models with both inference techniques were developed for total crashes; semi-parametric models were proved to provide substantial better model goodness-of-fit, while the two models shared consistent coefficient estimations. Later on, Bayesian semi-parametric random effects logistic regression models were developed for weekday peak hour crashes, weekday non-peak hour crashes, and weekend non-peak hour crashes to investigate different crash occurrence scenarios. Significant factors that affect crash risk have been revealed and crash mechanisms have been concluded.     

Bayesian analysis for zero-inflated regression models with the power prior: Applications to road safety countermeasures

We consider zero-inflated Poisson and zero-inflated negative binomial regression models to analyze discrete count data containing a considerable amount of zero observations. Analysis of current data could be empirically feasible if we utilize similar data based on previous studies. Ibrahim and Chen (2000) proposed the power prior to incorporate certain information from the historical data available from previous studies. The power prior is constructed by raising the likelihood function of the historical data to the power a₀, where 0≤a₀≤1. The power prior is a useful informative prior in Bayesian inference. We estimate regression coefficients associated with several safety countermeasures. We use Markov chain and Monte Carlo techniques to execute some computations. The empirical results show that the zero-inflated models with the power prior perform better than the frequentist approach.     

Simulation metamodelling with neural networks: An experimental investigation

Artificial neural networks are often proposed as an alternative approach for formalizing various quantitative and qualitative aspects of complex systems. This paper examines the robustness of using neural networks as a simulation metamodel to estimate manufacturing system performances. Simulation models of a job shop system are developed for various configurations to train neural network metamodels. Extensive computational tests are carried out with the proposed models at various factor levels (study horizon, system load, initial system status, stochasticity, system size and error assessment methods) to see the metamodel accuracy. The results indicate that simulation metamodels with neural networks can be effectively used to estimate the system performances.     

Investigating different approaches to develop informative priors in hierarchical Bayesian safety performance functions

The Bayesian inference method has been frequently adopted to develop safety performance functions. One advantage of the Bayesian inference is that prior information for the independent variables can be included in the inference procedures. However, there are few studies that discussed how to formulate informative priors for the independent variables and evaluated the effects of incorporating informative priors in developing safety performance functions. This paper addresses this deficiency by introducing four approaches of developing informative priors for the independent variables based on historical data and expert experience. Merits of these informative priors have been tested along with two types of Bayesian hierarchical models (Poisson-gamma and Poisson-lognormal models). Deviance information criterion (DIC), R-square values, and coefficients of variance for the estimations were utilized as evaluation measures to select the best model(s). Comparison across the models indicated that the Poisson-gamma model is superior with a better model fit and it is much more robust with the informative priors. Moreover, the two-stage Bayesian updating informative priors provided the best goodness-of-fit and coefficient estimation accuracies. Furthermore, informative priors for the inverse dispersion parameter have also been introduced and tested. Different types of informative priors’ effects on the model estimations and goodness-of-fit have been compared and concluded. Finally, based on the results, recommendations for future research topics and study applications have been made.     

Bayesian Additive Regression Tree Calibration of Complex High-Dimensional Computer Models

Complex natural phenomena are increasingly investigated by the use of a complex computer simulator. To leverage the advantages of simulators, observational data need to be incorporated in a probabilistic framework so that uncertainties can be quantified. A popular framework for such experiments is the statistical computer model calibration experiment. A limitation often encountered in current statistical approaches for such experiments is the difficulty in modeling high-dimensional observational datasets and simulator outputs as well as high-dimensional inputs. As the complexity of simulators seems to only grow, this challenge will continue unabated. In this article, we develop a Bayesian statistical calibration approach that is ideally suited for such challenging calibration problems. Our approach leverages recent ideas from Bayesian additive regression Tree models to construct a random basis representation of the simulator outputs and observational data. The approach can flexibly handle high-dimensional datasets, high-dimensional simulator inputs, and calibration parameters while quantifying important sources of uncertainty in the resulting inference. We demonstrate our methodology on a CO₂ emissions rate calibration problem, and on a complex simulator of subterranean radionuclide dispersion, which simulates the spatial–temporal diffusion of radionuclides released during nuclear bomb tests at the Nevada Test Site. Supplementary computer code and datasets are available online.     

Detection and characterization of chemotaxis without cell tracking

Swarming has been observed in various biological systems from collective animal movements to immune cells. In the cellular context, swarming is driven by the secretion of chemotactic factors. Despite the critical role of chemotactic swarming, few methods to robustly identify and quantify this phenomenon exist. Here, we present a novel method for the analysis of time series of positional data generated from realizations of agent-based processes. We convert the positional data for each individual time point to a function measuring agent aggregation around a given area of interest, hence generating a functional time series. The functional time series, and a more easily visualized swarming metric of agent aggregation derived from these functions, provide useful information regarding the evolution of the underlying process over time. We extend our method to build upon the modelling of collective motility using drift–diffusion partial differential equations (PDEs). Using a functional linear model, we are able to use the functional time series to estimate the drift and diffusivity terms associated with the underlying PDE. By producing an accurate estimate for the drift coefficient, we can infer the strength and range of attraction or repulsion exerted on agents, as in chemotaxis. Our approach relies solely on using agent positional data. The spatial distribution of diffusing chemokines is not required, nor do individual agents need to be tracked over time. We demonstrate our approach using random walk simulations of chemotaxis and experiments investigating cytotoxic T cells interacting with tumouroids.     

Timing HIV infection with a simple and accurate population viral dynamics model

Clinical trials for HIV prevention can require knowledge of infection times to subsequently determine protective drug levels. Yet, infection timing is difficult when study visits are sparse. Using population nonlinear mixed-effects (pNLME) statistical inference and viral loads from 46 RV217 study participants, we developed a relatively simple HIV primary infection model that achieved an excellent fit to all data. We also discovered that Aptima assay values from the study strongly correlated with viral loads, enabling imputation of very early viral loads for 28/46 participants. Estimated times between infecting exposures and first positives were generally longer than prior estimates (average of two weeks) and were robust to missing viral upslope data. On simulated data, we found that tighter sampling before diagnosis improved estimation more than tighter sampling after diagnosis. Sampling weekly before and monthly after diagnosis was a pragmatic design for good timing accuracy. Our pNLME timing approach is widely applicable to other infections with existing mathematical models. The present model could be used to simulate future HIV trials and may help estimate protective thresholds from the recently completed antibody-mediated prevention trials.     

Modelling and predicting the effect of social distancing and travel restrictions on COVID-19 spreading

To date, the only effective means to respond to the spreading of the COVID-19 pandemic are non-pharmaceutical interventions (NPIs), which entail policies to reduce social activity and mobility restrictions. Quantifying their effect is difficult, but it is key to reducing their social and economic consequences. Here, we introduce a meta-population model based on temporal networks, calibrated on the COVID-19 outbreak data in Italy and applied to evaluate the outcomes of these two types of NPIs. Our approach combines the advantages of granular spatial modelling of meta-population models with the ability to realistically describe social contacts via activity-driven networks. We focus on disentangling the impact of these two different types of NPIs: those aiming at reducing individuals’ social activity, for instance through lockdowns, and those that enforce mobility restrictions. We provide a valuable framework to assess the effectiveness of different NPIs, varying with respect to their timing and severity. Results suggest that the effects of mobility restrictions largely depend on the possibility of implementing timely NPIs in the early phases of the outbreak, whereas activity reduction policies should be prioritized afterwards.