Integrated mechanistic and data-driven modelling for multivariate analysis of signalling pathways.

Mathematical models of highly interconnected and multivariate signalling networks provide useful tools to understand these complex systems. However, effective approaches to extracting multivariate regulation information from these models are still lacking. In this study, we propose a data-driven modelling framework to analyse large-scale multivariate datasets generated from mathematical models. We used an ordinary differential equation based model for the Fas apoptotic pathway as an example. The first step in our approach was to cluster simulation outputs generated from models with varied protein initial concentrations. Subsequently, decision tree analysis was applied, in which we used protein concentrations to predict the simulation outcomes. Our results suggest that no single subset of proteins can determine the pathway behaviour. Instead, different subsets of proteins with different concentrations ranges can be important. We also used the resulting decision tree to identify the minimal number of perturbations needed to change pathway behaviours. In conclusion, our framework provides a novel approach to understand the multivariate dependencies among molecules in complex networks, and can potentially be used to identify combinatorial targets for therapeutic interventions.     

The use of LS–PLS for improved understanding, monitoring and prediction of cheese processing

In this paper we demonstrate both a design strategy and a set of analysis techniques for a designed experiment from an industrial process (cheese making) with multivariate responses (sensory data). The design strategy uses two-level factorial design for the factors that can be controlled, and blocking on the raw material to cover other non-designed variation in the raw material. We measure both the raw materials and on several points during the process with FT-IR spectroscopy. The methods of analysis complement each other to give more understanding and better modelling. The 50–50 MANOVA method provides multivariate analysis of variance to test for significance of effects for the design variables. Ordinary PLS2 analysis gives an overview of the data and generates hypotheses about relations. Finally, the orthogonal LS–PLS method is extended to multivariate responses and used to identify the source of the observed block effect and to build models that can be used for statistical process control at several points in the process. In these models, the information at one point is corrected for information that has already been described elsewhere.     

Two Projection Methods for Use in the Analysis of Multivariate Process Data With an Illustration in Petrochemical Production

Principal components analysis (PCA) is often used in the analysis of multivariate process data to identify important combinations of the original variables on which to focus for more detailed study. However, PCA and other related projection techniques from the standard multivariate repertoire are not explicitly designed to address or to exploit the strong autocorrelation and temporal cross-correlation structures that are often present in multivariate process data. Here we propose two alternative projection techniques that do focus on the temporal structure in such data and that therefore produce components that may have some analytical advantages over those resulting from more conventional multivariate methods. As in PCA, both of our suggested methods linearly transform the original p-variate time series into uncorrelated components; however, unlike PCA, they concentrate on deriving components with particular temporal correlation properties, rather than those with maximal variance. The first technique finds components that exhibit distinctly different autocorrelation structures via modification of a signal-noise decomposition method used in image analysis. The second method draws on ideas from common PCA to produce components that are not only uncorrelated as in PCA, but that also have approximately zero temporally lagged cross-correlations for all time lags. We present the technical details for these two methods, assess their performance through simulation studies, and illustrate their use on multivariate output measures from a fluidized catalytic cracking unit used in petrochemical production, contrasting the results obtained with those from standard PCA.     

Expanding the Scope of Multivariate Regression Approaches in Cross-Omics Research

Recent technological advancements and developments have led to a dramatic increase in the amount of high-dimensional data and thus have increased the demand for proper and efficient multivariate regression methods. Numerous traditional multivariate approaches such as principal component analysis have been used broadly in various research areas, including investment analysis, image identification, and population genetic structure analysis. However, these common approaches have the limitations of ignoring the correlations between responses and a low variable selection efficiency. Therefore, in this article, we introduce the reduced rank regression method and its extensions, sparse reduced rank regression and subspace assisted regression with row sparsity, which hold potential to meet the above demands and thus improve the interpretability of regression models. We conducted a simulation study to evaluate their performance and compared them with several other variable selection methods. For different application scenarios, we also provide selection suggestions based on predictive ability and variable selection accuracy. Finally, to demonstrate the practical value of these methods in the field of microbiome research, we applied our chosen method to real population-level microbiome data, the results of which validated our method. Our method extensions provide valuable guidelines for future omics research, especially with respect to multivariate regression, and could pave the way for novel discoveries in microbiome and related research fields.     

Dynamic bounds coupled with Monte Carlo simulations

For the reliability analysis of engineering structures a variety of methods is known, of which Monte Carlo (MC) simulation is widely considered to be among the most robust and most generally applicable. To reduce simulation cost of the MC method, variance reduction methods are applied. This paper describes a method to reduce the simulation cost even further, while retaining the accuracy of Monte Carlo, by taking into account widely present monotonicity. For models exhibiting monotonic (decreasing or increasing) behavior, dynamic bounds (DB) are defined, which in a coupled Monte Carlo simulation are updated dynamically, resulting in a failure probability estimate, as well as a strict (non-probabilistic) upper and lower bounds. Accurate results are obtained at a much lower cost than an equivalent ordinary Monte Carlo simulation. In a two-dimensional and a four-dimensional numerical example, the cost reduction factors are 130 and 9, respectively, where the relative error is smaller than 5%. At higher accuracy levels, this factor increases, though this effect is expected to be smaller with increasing dimension. To show the application of DB method to real world problems, it is applied to a complex finite element model of a flood wall in New Orleans.     

Kernel principal component analysis and support vector machines for stock price prediction

Technical indicators are used with two heuristic models, kernel principal component analysis and factor analysis in order to identify the most influential inputs for a forecasting model. Multilayer perceptron (MLP) networks and support vector regression (SVR) are used with different inputs. We assume that the future value of a stock price/return depends on the financial indicators although there is no parametric model to explain this relationship, which comes from the technical analysis. Comparison studies show that SVR and MLP networks require different inputs. Furthermore, proposed heuristic models produce better results than the studied data mining methods. In addition to this, we can say that there is no difference between MLP networks and SVR techniques when we compare their mean square error values.     

Efficient Integration of Sufficient Dimension Reduction and Prediction in Discriminant Analysis

Sufficient dimension reduction (SDR) methods are popular model-free tools for preprocessing and data visualization in regression problems where the number of variables is large. Unfortunately, reduce-and-classify approaches in discriminant analysis usually cannot guarantee improvement in classification accuracy, mainly due to the different nature of the two stages. On the other hand, envelope methods construct targeted dimension reduction subspaces that achieve dimension reduction and improve parameter estimation efficiency at the same time. However, little is known about how to construct envelopes in discriminant analysis models. In this article, we introduce the notion of the envelope discriminant subspace (ENDS) as a natural inferential and estimative object in discriminant analysis that incorporates these considerations. We develop the ENDS estimators that simultaneously achieve sufficient dimension reduction and classification. Consistency and asymptotic normality of the ENDS estimators are established, where we carefully examine the asymptotic efficiency gain under the classical linear and quadratic discriminant analysis models. Simulations and real data examples show superb performance of the proposed method. Supplementary materials for this article are available online.     

Distribution-free high-dimensional two-sample tests based on discriminating hyperplanes

In this article, we propose a general procedure for multivariate generalizations of univariate distribution-free tests involving two independent samples as well as matched pair data. This proposed procedure is based on ranks of real-valued linear functions of multivariate observations. The linear function used to rank the observations is obtained by solving a classification problem between the two multivariate distributions from which the observations are generated. Our proposed tests retain the distribution-free property of their univariate analogs, and they perform well for high-dimensional data even when the dimension exceeds the sample size. Asymptotic results on their power properties are derived when the dimension grows to infinity and the sample size may or may not grow with the dimension. We analyze several high-dimensional simulated and real data sets to compare the empirical performance of our proposed tests with several other tests available in the literature.     

结构复杂行为分析的有限质点法研究综述

有限质点法是一种结构分析的新方法,它以向量力学理论和数值计算为基础,以点值描述和途径单元为基本概念,以清晰的物理模型和质点运动控制方程描述结构行为。该方法的计算不需组集单元的刚度矩阵,也不需迭代求解控制方程式。与传统方法相比,在结构的动力、几何非线性、材料非线性、屈曲或褶皱失效、机构运动、接触和碰撞等复杂行为分析中有较大的优势。该文首先介绍有限质点法的基本理论,在此基础上着重阐述这种新的数值分析方法在空间结构复杂行为研究领域的优势及应用,并对该方法的发展趋势作出展望。     

Exploring Collective Dynamics in Communication Networks

A communication network, such as the Internet, comprises a complex system where cooperative phenomena may emerge from interactions among various traffic flows generated and forwarded by individual nodes. To identify and understand such phenomena, we model a network as a two-dimensional cellular automaton. We suspect such models can promote better understanding of the spatial-temporal evolution of network congestion, and other emergent phenomena in communication networks. To search the behavior space of the model, we study dynamic patterns arising from interactions among traffic flows routed across shared network nodes, as we employ various configurations of parameters and two different congestion-control algorithms. In this paper, we characterize correlation in congestion behavior within the model at different system sizes and time granularities. As expected, we find that long-range dependence (LRD) appears at some time granularities, and that for a given network size LRD decays as time granularity increases. As network size increases, we find that long-range dependence exists at larger time scales. To distinguish effects due to network size from effects due to collective phenomena, we compare congestion behavior within networks of selected sizes to congestion behavior within comparably sized sub-areas in a larger network. We find stronger long-range dependence for sub-areas within the larger network. This suggests the importance of modeling networks of sufficiently large size when studying the effects of collective dynamics.     

基于多元半参数回归建模的机械加工过程误差分析

机械加工过程的误差分析对于消减误差源、提升工序质量具有重要的实际指导意义．研究者们尝试了多种误差分析方法用于获得误差源对加工质量的作用规律,然而由于加工误差的复杂性,各有局限性．本文则根据工程经验和数学推导建立了用于一般机械加工过程误差分析的多元半参数回归模型,基于测量数据详细讨论了所建立模型的参数估计和非参数规律辨识问题．仿真实例证明,与现有方法相比,本文所提方法能够准确估计上游工序传递误差,有效辨识当前工序系统误差的作用规律,研究成果为一般机械加工过程的误差分析奠定了基础．     

Digital clocks: simple Boolean models can quantitatively describe circadian systems

The gene networks that comprise the circadian clock modulate biological function across a range of scales, from gene expression to performance and adaptive behaviour. The clock functions by generating endogenous rhythms that can be entrained to the external 24-h day–night cycle, enabling organisms to optimally time biochemical processes relative to dawn and dusk. In recent years, computational models based on differential equations have become useful tools for dissecting and quantifying the complex regulatory relationships underlying the clock''s oscillatory dynamics. However, optimizing the large parameter sets characteristic of these models places intense demands on both computational and experimental resources, limiting the scope of in silico studies. Here, we develop an approach based on Boolean logic that dramatically reduces the parametrization, making the state and parameter spaces finite and tractable. We introduce efficient methods for fitting Boolean models to molecular data, successfully demonstrating their application to synthetic time courses generated by a number of established clock models, as well as experimental expression levels measured using luciferase imaging. Our results indicate that despite their relative simplicity, logic models can (i) simulate circadian oscillations with the correct, experimentally observed phase relationships among genes and (ii) flexibly entrain to light stimuli, reproducing the complex responses to variations in daylength generated by more detailed differential equation formulations. Our work also demonstrates that logic models have sufficient predictive power to identify optimal regulatory structures from experimental data. By presenting the first Boolean models of circadian circuits together with general techniques for their optimization, we hope to establish a new framework for the systematic modelling of more complex clocks, as well as other circuits with different qualitative dynamics. In particular, we anticipate that the ability of logic models to provide a computationally efficient representation of system behaviour could greatly facilitate the reverse-engineering of large-scale biochemical networks.     

A new Bayesian multivariate exponentially weighted moving average control chart for phase II monitoring of multivariate multiple linear profiles

Recently, statistical profile monitoring methods have become efficient tools for monitoring the quality of a product (or a production process) using control charts. The key idea is to describe the relationship between a response variable and a set of explanatory variables in the form of a statistical regression model, which called profile. Traditionally, those control charts are constructed with standard “frequentistic” regression models. Recently, it has been proposed to apply Bayesian regression models instead, and it has been empirically demonstrated that Bayesian regression models have the potential to perform significantly better. In this paper, we introduce a novel Bayesian multivariate exponentially weighted moving average control chart for monitoring multivariate multiple linear profiles in phase II. The key idea is to use the data from historical data sets to generate informative prior distributions for the regression models in phase II. The results of our empirical simulation studies show that the Bayesian multivariate multiple linear regression model is superior to its classical “frequentistic” counterpart in terms of the average run length. Our empirical findings are in agreement with findings reported in recently published articles. To shed more light onto the merit of the proposed Bayesian method, we carry out a sensitivity analysis, in which we investigate how the amount of phase I data influences the results. We also demonstrate the applicability and superiority of the proposed Bayesian method by a real‐world application.     

Bayesian analysis for estimating statistical parameter distributions of elasto-viscoplastic material models

High temperature design methods rely on constitutive models for inelastic deformation and failure typically calibrated against the mean of experimental data without considering the associated scatter. Variability may arise from the experimental data acquisition process, from heat-to-heat material property variations, or both and need to be accurately captured to predict parameter bounds leading to efficient component design. Applying the Bayesian Markov Chain Monte Carlo (MCMC) method to produce statistical models capturing the underlying uncertainty in the experimental data is an area of ongoing research interest. This work varies aspects of the Bayesian MCMC method and explores their effect on the posterior parameter distributions for a uniaxial elasto-viscoplastic damage model using synthetically generated reference data. From our analysis with the uniaxial inelastic model we determine that an informed prior distribution including different types of test conditions results in more accurate posterior parameter distributions. The parameter posterior distributions, however, do not improve when increasing the number of similar experimental data. Additionally, changing the amount of scatter in the data affects the quality of the posterior distributions, especially for the less sensitive model parameters. Moreover, we perform a sensitivity study of the model parameters against the likelihood function prior to the Bayesian analysis. The results of the sensitivity analysis help to determine the reliability of the posterior distributions and reduce the dimensionality of the problem by fixing the insensitive parameters. The comprehensive study described in this work demonstrates how to efficiently apply the Bayesian MCMC methodology to capture parameter uncertainties in high temperature inelastic material models. Quantifying these uncertainties in inelastic models will improve high temperature engineering design practices and lead to safer, more effective component designs.     

Using modified nodal analysis in cavity-mode resonances printed circuit board power bus structures with the segmentation method

In this paper, we used the modified nodal analysis (MNA) method to obtain the cavity-mode equation for the segmentation method. In the experiment, the calculation result of MNA is more accurate than traditional double summation with discrete capacitors. The proposed approaches decrease computation and easily add discrete capacitors. The MNA and segmentation methods can be combined to conveniently calculate an irregularly shaped cavity by using computers. This approach can reduce the time required for deriving the impedance equation. The result can be used to build a mathematical model of the MNA method, which can suppress cavity-mode resonances within the power bus by using discrete capacitors. We used electromagnetic simulation software to verify these models.     

Thresholded Multivariate Principal Component Analysis for Phase I Multichannel Profile Monitoring

Monitoring multichannel profiles has important applications in manufacturing systems improvement, but it is nontrivial to develop efficient statistical methods because profiles are high-dimensional functional data with intrinsic inner- and interchannel correlations, and that the change might only affect a few unknown features of multichannel profiles. To tackle these challenges, we propose a novel thresholded multivariate principal component analysis (PCA) method for multichannel profile monitoring. Our proposed method consists of two steps of dimension reduction: It first applies the functional PCA to extract a reasonably large number of features under the in-control state, and then uses the soft-thresholding techniques to further select significant features capturing profile information under the out-of-control state. The choice of tuning parameter for soft-thresholding is provided based on asymptotic analysis, and extensive numerical studies are conducted to illustrate the efficacy of our proposed thresholded PCA methodology.     

A Unified Analysis of Structured Sonar-Terrain Data Using Bayesian Functional Mixed Models

Sonar emits pulses of sound and uses the reflected echoes to gain information about target objects. It offers a low cost, complementary sensing modality for small robotic platforms. Although existing analytical approaches often assume independence across echoes, real sonar data can have more complicated structures due to device setup or experimental design. In this article, we consider sonar echo data collected from multiple terrain substrates with a dual-channel sonar head. Our goals are to identify the differential sonar responses to terrains and study the effectiveness of this dual-channel design in discriminating targets. We describe a unified analytical framework that achieves these goals rigorously, simultaneously, and automatically. The analysis was done by treating the echo envelope signals as functional responses and the terrain/channel information as covariates in a functional regression setting. We adopt functional mixed models that facilitate the estimation of terrain and channel effects while capturing the complex hierarchical structure in data. This unified analytical framework incorporates both Gaussian models and robust models. We fit the models using a full Bayesian approach, which enables us to perform multiple inferential tasks under the same modeling framework, including selecting models, estimating the effects of interest, identifying significant local regions, discriminating terrain types, and describing the discriminatory power of local regions. Our analysis of the sonar-terrain data identifies time regions that reflect differential sonar responses to terrains. The discriminant analysis suggests that a multi- or dual-channel design achieves target identification performance comparable with or better than a single-channel design. Supplementary materials for this article are available online.     

Large-Vector Autoregression for Multilayer Spatially Correlated Time Series

One of the most commonly used methods for modeling multivariate time series is the vector autoregressive model (VAR). VAR is generally used to identify lead, lag, and contemporaneous relationships describing Granger causality within and between time series. In this article, we investigate the VAR methodology for analyzing data consisting of multilayer time series that are spatially interdependent. When modeling VAR relationships for such data, the dependence between time series is both a curse and a blessing. The former because it requires modeling the between time-series correlation or the contemporaneous relationships which may be challenging when using likelihood-based methods. The latter because the spatial correlation structure can be used to specify the lead–lag relationships within and between time series, within and between layers. To address these challenges, we propose an L₁\L₂ regularized likelihood estimation method. The lead, lag, and contemporaneous relationships are estimated using an efficient algorithm that exploits sparsity in the VAR structure, accounts for the spatial dependence, and models the error dependence. We consider a case study to illustrate the applicability of our method. In the supplementary materials available online, we assess the performance of the proposed VAR model and compare it with existing methods within a simulation study.