Sensitivity analysis approaches applied to systems biology models

With the rising application of systems biology, sensitivity analysis methods have been widely applied to study the biological systems, including metabolic networks, signalling pathways and genetic circuits. Sensitivity analysis can provide valuable insights about how robust the biological responses are with respect to the changes of biological parameters and which model inputs are the key factors that affect the model outputs. In addition, sensitivity analysis is valuable for guiding experimental analysis, model reduction and parameter estimation. Local and global sensitivity analysis approaches are the two types of sensitivity analysis that are commonly applied in systems biology. Local sensitivity analysis is a classic method that studies the impact of small perturbations on the model outputs. On the other hand, global sensitivity analysis approaches have been applied to understand how the model outputs are affected by large variations of the model input parameters. In this review, the author introduces the basic concepts of sensitivity analysis approaches applied to systems biology models. Moreover, the author discusses the advantages and disadvantages of different sensitivity analysis methods, how to choose a proper sensitivity analysis approach, the available sensitivity analysis tools for systems biology models and the caveats in the interpretation of sensitivity analysis results.     

Effective sampling trajectory optimisation for sensitivity analysis of biological systems

Sensitivity analysis has been widely applied to study the biological systems, including metabolic networks, signalling pathways, and genetic circuits. The Morris method is a kind of screening sensitivity analysis approach, which can fast identify a few key factors from numerous biological parameters and inputs. The parameter or input space is randomly sampled to produce a very limited number of trajectories for the calculation of elementary effects. It is clear that the sampled trajectories are not enough to cover the whole uncertain space, which eventually causes unstable sensitivity measures. This paper presents a novel trajectory optimisation algorithm for the Morris‐based sensitivity calculation to ensure a good scan throughout the whole uncertain space. The paper demonstrates that this presented method gets more consistent sensitivity results through a benchmark example. The application to a previously published ordinary differential equation model of a cellular signalling network is presented. In detail, the parameter sensitivity analysis verifies the good agreement with data of the literatures.Inspec keywords: genetics, differential equations, sensitivity analysis, biology, sampling methods, optimisationOther keywords: biological systems, metabolic networks, genetic circuits, Morris‐based sensitivity calculation, ordinary differential equation, sampling trajectory optimisation, sensitivity analysis, parameter sensitivity analysis, cellular signalling network     

Inferring catalysis in biological systems

In systems biology, one is often interested in the communication patterns between several species, such as genes, enzymes or proteins. These patterns become more recognisable when temporal experiments are performed. This temporal communication can be structured by reaction networks such as gene regulatory networks or signalling pathways. Mathematical modelling of data arising from such networks can reveal important details, thus helping to understand the studied system. In many cases, however, corresponding models still deviate from the observed data. This may be due to unknown but present catalytic reactions. From a modelling perspective, the question of whether a certain reaction is catalysed leads to a large increase of model candidates. For large networks the calibration of all possible models becomes computationally infeasible. We propose a method which determines a substantially reduced set of appropriate model candidates and identifies the catalyst of each reaction at the same time. This is incorporated in a multiple‐step procedure which first extends the network by additional latent variables and subsequently identifies catalyst candidates using similarity analysis methods. Results from synthetic data examples suggest a good performance even for non‐informative data with few observations. Applied on CD95 apoptotic pathway our method provides new insights into apoptosis regulation.Inspec keywords: catalysis, catalysts, biochemistry, genetics, enzymes, biology computing, calibration, molecular clustersOther keywords: inferring catalysis, biological systems, systems biology, communication patterns, genes, enzymes, proteins, time‐resolved experiments, time‐resolved communication, reaction networks, gene regulatory networks, biochemical networks, signalling pathways, mathematical data modelling, catalytic reactions, calibration, catalyst, multiple‐step procedure, latent variables, similarity analysis methods, noninformative data, differentiation apoptotic pathway, cluster     

Identifying latent dynamic components in biological systems

In computational systems biology, the general aim is to derive regulatory models from multivariate readouts, thereby generating predictions for novel experiments. In the past, many such models have been formulated for different biological applications. The authors consider the scenario where a given model fails to predict a set of observations with acceptable accuracy and ask the question whether this is because of the model lacking important external regulations. Real‐world examples for such entities range from microRNAs to metabolic fluxes. To improve the prediction, they propose an algorithm to systematically extend the network by an additional latent dynamic variable which has an exogenous effect on the considered network. This variable''s time course and influence on the other species is estimated in a two‐step procedure involving spline approximation, maximum‐likelihood estimation and model selection. Simulation studies show that such a hidden influence can successfully be inferred. The method is also applied to a signalling pathway model where they analyse real data and obtain promising results. Furthermore, the technique can be employed to detect incomplete network structures.Inspec keywords: biology computing, RNA, splines (mathematics), maximum likelihood estimation, approximation theory, biochemistryOther keywords: latent dynamic components, biological systems, computational system biology, regulatory models, multivariate readouts, biological applications, external regulations, real‐world examples, microRNA, metabolic fluxes, latent dynamic variables, variable time course, two‐step procedure, spline approximation, maximum‐likelihood estimation, model selection, signalling pathway model, real data, incomplete network structures     

A meta-heuristic cuckoo search and eigen permutation approach for model order reduction

In this paper, a new model order reduction technique is presented by combining the benefits of the meta-heuristic cuckoo search optimization and Eigen permutation methods for order reduction of higher order continuous-time systems. In the proposed approach, the numerator and the denominator polynomials of reduced order model are determined by Cuckoo search and Eigen permutation approaches, respectively. The proposed approach preserves the stability of the original system into the lower order model as the Eigen permutation retains the dominant pole with simultaneous cluster formation of the remaining real and complex poles. The effectiveness of the proposed method is validated by single-input single-output and multiple-inputs multiple-outputs numerical examples.     

Spectral radiative-transfer modeling with minimized computation time by use of a neural-network technique

A new approach based on a neural-network technique for reduction in the computation time of radiative-transfer models is presented. This approach gives high spectral resolution without significant loss of accuracy. A rigorous radiative-transfer model is used to calculate radiation values at a few selected wavelengths, and a neural-network algorithm replenishes them to a complete spectrum with radiation values at a high spectral resolution. This method is used for the UV and visible spectral ranges. The results document the ability of a neural network to learn this specific task. More than 20,000 UV-index values for all kinds of atmosphere are calculated by both the rigorous radiative-transfer model alone and the model in combination with the neural-network algorithm. The agreement between both approaches is generally of the order of ?1%; the computation time is reduced by a factor of more than 20. The new algorithm can be used for all kinds of high-quality radiative-transfer model to speed up computation time.     

Order reduction of linear time-invariant systems using Eigen permutation and Jaya algorithm

This article presents a new mixed method for order reduction of higher order linear time-invariant systems using Eigen permutation and the Jaya optimization algorithm. The Jaya algorithm is an algorithm-specific parameter-free optimization which requires tuning of standard control parameters only. On the other hand, the Eigen permutation retains the dominant poles of the original system with simultaneous cluster formation of the remaining real and complex poles. The numerator and the denominator polynomials of the reduced order model are determined using the Jaya optimization and Eigen permutation approaches, respectively. The proposed method results in a stable reduced order model for a stable higher order system. The effectiveness of the proposed method is validated using numerical examples of single-input–single-output and multiple-input–multiple-output systems. Furthermore, the results are compared with well-known existing techniques available in the literature.     

Using Krylov‐Padé model order reduction for accelerating design optimization of structures and vibrations in the frequency domain

In many engineering problems, the behavior of dynamical systems depends on physical parameters. In design optimization, these parameters are determined so that an objective function is minimized. For applications in vibrations and structures, the objective function depends on the frequency response function over a given frequency range, and we optimize it in the parameter space. Because of the large size of the system, numerical optimization is expensive. In this paper, we propose the combination of Quasi‐Newton type line search optimization methods and Krylov‐Padé type algebraic model order reduction techniques to speed up numerical optimization of dynamical systems. We prove that Krylov‐Padé type model order reduction allows for fast evaluation of the objective function and its gradient, thanks to the moment matching property for both the objective function and the derivatives towards the parameters. We show that reduced models for the frequency alone lead to significant speed ups. In addition, we show that reduced models valid for both the frequency range and a line in the parameter space can further reduce the optimization time. Copyright © 2012 John Wiley & Sons, Ltd.     

Hessian‐based model reduction for large‐scale systems with initial‐condition inputs

Reduced‐order models that are able to approximate output quantities of interest of high‐fidelity computational models over a wide range of input parameters play an important role in making tractable large‐scale optimal design, optimal control, and inverse problem applications. We consider the problem of determining a reduced model of an initial value problem that spans all important initial conditions, and pose the task of determining appropriate training sets for reduced‐basis construction as a sequence of optimization problems. We show that, under certain assumptions, these optimization problems have an explicit solution in the form of an eigenvalue problem, yielding an efficient model reduction algorithm that scales well to systems with states of high dimension. Furthermore, tight upper bounds are given for the error in the outputs of the reduced models. The reduction methodology is demonstrated for a large‐scale contaminant transport problem. Copyright © 2007 John Wiley & Sons, Ltd.     

Multi-element probabilistic collocation for sensitivity analysis in cellular signalling networks

The multi-element probabilistic collocation method (ME-PCM) as a tool for sensitivity analysis of differential equation models as applied to cellular signalling networks is formulated. This method utilises a simple, efficient sampling algorithm to quantify local sensitivities throughout the parameter space. The application of the ME-PCM to a previously published ordinary differential equation model of the apoptosis signalling network is presented. The authors verify agreement with the previously identified regions of sensitivity and then go on to analyse this region in greater detail with the ME-PCM. The authors demonstrate the generality of the ME-PCM by studying sensitivity of the system using a variety of biologically relevant markers in the system such as variation in one (or many) chemical species as a function of time, and total exposure of a single chemical species.     

Model reduction for LPV systems based on approximate modal decomposition

The paper presents a novel model order reduction technique for large‐scale linear parameter‐varying (LPV) systems. The approach is based on decoupling the original dynamics into smaller dimensional LPV subsystems that can be independently reduced by parameter‐varying reduction methods. The decomposition starts with the construction of a modal transformation that separates the modal subsystems. Hierarchical clustering is applied then to collect the dynamically similar modal subsystems into larger groups. The resulting parameter‐varying subsystems are then independently reduced. This approach substantially differs from most of the previously proposed LPV model reduction techniques, since it performs manipulations on the LPV model itself, instead of on a set of linear time‐invariant models defined at fixed scheduling parameter values. Therefore, the interpolation, which is often a challenging part in reduction techniques, is inherently solved. The applicability of the developed algorithm is thoroughly investigated and demonstrated by numerical case studies.     

Effective connectivity determines the critical dynamics of biochemical networks

Living systems comprise interacting biochemical components in very large networks. Given their high connectivity, biochemical dynamics are surprisingly not chaotic but quite robust to perturbations—a feature C.H. Waddington named canalization. Because organisms are also flexible enough to evolve, they arguably operate in a critical dynamical regime between order and chaos. The established theory of criticality is based on networks of interacting automata where Boolean truth values model presence/absence of biochemical molecules. The dynamical regime is predicted using network connectivity and node bias (to be on/off) as tuning parameters. Revising this to account for canalization leads to a significant improvement in dynamical regime prediction. The revision is based on effective connectivity, a measure of dynamical redundancy that buffers automata response to some inputs. In both random and experimentally validated systems biology networks, reducing effective connectivity makes living systems operate in stable or critical regimes even though the structure of their biochemical interaction networks predicts them to be chaotic. This suggests that dynamical redundancy may be naturally selected to maintain living systems near critical dynamics, providing both robustness and evolvability. By identifying how dynamics propagates preferably via effective pathways, our approach helps to identify precise ways to design and control network models of biochemical regulation and signalling.     

A distributed algorithm for fitting generalized additive models

Generalized additive models are an effective regression tool, popular in the statistics literature, that provides an automatic extension of traditional linear models to nonlinear systems. We present a distributed algorithm for fitting generalized additive models, based on the alternating direction method of multipliers (ADMM). In our algorithm the component functions of the model are fit independently, in parallel; a simple iteration yields convergence to the optimal generalized additive model. This is in contrast to the traditional approach of backfitting, where the component functions are fit sequentially. We illustrate the method on different classes of problems such as generalized additive, logistic, and piecewise constant models, with various types of regularization, including those that promote smoothness and sparsity.     

Exact model reduction with delays: closed-form distributions and extensions to fully bi-directional monomolecular reactions

In order to systematically understand the qualitative and quantitative behaviour of chemical reaction networks, scientists must derive and analyse associated mathematical models. However, biochemical systems are often very large, with reactions occurring at multiple time scales, as evidenced by signalling pathways and gene expression kinetics. Owing to the associated computational costs, it is then many times impractical, if not impossible, to solve or simulate these systems with an appropriate level of detail. By consequence, there is a growing interest in developing techniques for the simplification or reduction of complex biochemical systems. Here, we extend our recently presented methodology on exact reduction of linear chains of reactions with delay distributions in two ways. First, we report that it is now possible to deal with fully bi-directional monomolecular systems, including degradations, synthesis and generalized bypass reactions. Second, we provide all derivations of associated delays in analytical, closed form. Both advances have a major impact on further reducing computational costs, while still retaining full accuracy. Thus, we expect our new methodology to respond to current simulation needs in pharmaceutical, chemical and biological research.     

Accelerated mesh sampling for the hyper reduction of nonlinear computational models

In nonlinear model order reduction, hyper reduction designates the process of approximating a projection‐based reduced‐order operator on a reduced mesh, using a numerical algorithm whose computational complexity scales with the small size of the projection‐based reduced‐order model. Usually, the reduced mesh is constructed by sampling the large‐scale mesh associated with the high‐dimensional model underlying the projection‐based reduced‐order model. The sampling process itself is governed by the minimization of the size of the reduced mesh for which the hyper reduction method of interest delivers the desired accuracy for a chosen set of training reduced‐order quantities. Because such a construction procedure is combinatorially hard, its key objective function is conveniently substituted with a convex approximation. Nevertheless, for large‐scale meshes, the resulting mesh sampling procedure remains computationally intensive. In this paper, three different convex approximations that promote sparsity in the solution are considered for constructing reduced meshes that are suitable for hyper reduction and paired with appropriate active set algorithms for solving the resulting minimization problems. These algorithms are equipped with carefully designed parallel computational kernels in order to accelerate the overall process of mesh sampling for hyper reduction, and therefore achieve practicality for realistic, large‐scale, nonlinear structural dynamics problems. Conclusions are also offered as to what algorithm is most suitable for constructing a reduced mesh for the purpose of hyper reduction. Copyright © 2016 John Wiley & Sons, Ltd.     

Nonlinear gas turbine modeling using NARMAX structures

The estimation of a nonlinear autoregressive moving average with exogenous inputs (NARMAX) model of an aircraft gas turbine is presented. A method is proposed whereby periodic signals with certain harmonic content are used to qualify the nature of the nonlinearity of the engine in the frequency domain. The static behavior of the engine is investigated in the time domain to approximate the order of nonlinearity and this information is used a priori to restrict the search space of the potential NARMAX models. A forward-regression orthogonal estimation algorithm is then employed to select the model terms using the error reduction ratio. The performance of the estimated NARMAX model is illustrated against a range of small- and large-signal engine tests     

The Alpha Project: a model system for systems biology research

One goal of systems biology is to understand how genome-encoded parts interact to produce quantitative phenotypes. The Alpha Project is a medium-scale, interdisciplinary systems biology effort that aims to achieve this goal by understanding fundamental quantitative behaviours of a prototypic signal transduction pathway, the yeast pheromone response system from Saccharomyces cerevisiae. The Alpha Project distinguishes itself from many other systems biology projects by studying a tightly bounded and well-characterised system that is easily modified by genetic means, and by focusing on deep understanding of a discrete number of important and accessible quantitative behaviours. During the project, the authors have developed tools to measure the appropriate data and develop models at appropriate levels of detail to study a number of these quantitative behaviours. The authors have also developed transportable experimental tools and conceptual frameworks for understanding other signalling systems. In particular, the authors have begun to interpret system behaviours and their underlying molecular mechanisms through the lens of information transmission, a principal function of signalling systems. The Alpha Project demonstrates that interdisciplinary studies that identify key quantitative behaviours and measure important quantities, in the context of well-articulated abstractions of system function and appropriate analytical frameworks, can lead to deeper biological understanding. The authors' experience may provide a productive template for systems biology investigations of other cellular systems.     

Model order reduction for optimal bounding ellipsoid channel models

This paper presents a new algorithm for performing model order reduction for Volterra series channel models of high-density digital magnetic recording channels. We employ a set-membership approach to the problem in which a set of consistent modeling solutions bounded by an optimal ellipsoid is first developed for the channel. We then present a new algorithm for finding the minimum number of coefficients in the Volterra series expansion which preserves the accuracy, in a least-squares sense, of the reduced order model in comparison with the original model