Inference in HIV dynamics models via hierarchical likelihood

HIV dynamical models are often based on non-linear systems of ordinary differential equations (ODE), which do not have an analytical solution. Introducing random effects in such models leads to very challenging non-linear mixed-effects models. To avoid the numerical computation of multiple integrals involved in the likelihood, a hierarchical likelihood (h-likelihood) approach, treated in the spirit of a penalized likelihood is proposed. The asymptotic distribution of the maximum h-likelihood estimators (MHLE) for fixed effects is given. The MHLE are slightly biased but the bias can be made negligible by using a parametric bootstrap procedure. An efficient algorithm for maximizing the h-likelihood is proposed. A simulation study, based on a classical HIV dynamical model, confirms the good properties of the MHLE. The method is applied to the analysis of a clinical trial.     

Information methods for model selection in linear mixed effects models with application to HCV data

In this paper, we derive a small sample Akaike information criterion, based on the maximized loglikelihood, and a small sample information criterion based on the maximized restricted loglikelihood in the linear mixed effects model when the covariance matrix of the random effects is known. Small sample corrected information criteria are proposed for a special case of linear mixed effects models, the balanced random-coefficient model, without assuming the random coefficients covariance matrix to be known. A simulation study comparing the derived criteria and several others for model selection in the linear mixed effects models is presented. We illustrate the behavior of the studied information criteria on real data from a study of subjects coinfected with HIV and Hepatitis C virus. Robustness of the criteria, in terms of the error distributed as a mixture of normal distributions, is also studied. Special attention is given to the behavior of the conditional AIC by Vaida and Blanchard (2005). Among the studied criteria, GIC performs best, while cAIC exhibits poor performance. Because of its inferior performance, as demonstrated in this work, we do not recommend its use for model selection in linear mixed effects models.     

Bayesian Cholesky factor models in random effects covariance matrix for generalized linear mixed models

Random effects in generalized linear mixed models (GLMM) are used to explain the serial correlation of the longitudinal categorical data. Because the covariance matrix is high dimensional and should be positive definite, its structure is assumed to be constant over subjects and to be restricted such as AR(1) structure. However, these assumptions are too strong and can result in biased estimates of the fixed effects. In this paper we propose a Bayesian modeling for the GLMM with regression models for parameters of the random effects covariance matrix using a moving average Cholesky decomposition which factors the covariance matrix into moving average (MA) parameters and IVs. We analyze lung cancer data using our proposed model.     

Bayesian analysis of semiparametric reproductive dispersion mixed-effects models

Semiparametric reproductive dispersion mixed-effects model (SPRDMM) is an extension of the reproductive dispersion model and the semiparametric mixed model, and it includes many commonly encountered models as its special cases. A Bayesian procedure is developed for analyzing SPRDMMs on the basis of P-spline estimates of nonparametric components. A hybrid algorithm combining the Gibbs sampler and the Metropolis-Hastings algorithm is used to simultaneously obtain the Bayesian estimates of unknown parameters, smoothing function and random effects, as well as their standard error estimates. The Bayes factor for model comparison is employed to select better approximation of the smoothing function via path sampling. Several simulation studies and a real example are used to illustrate the proposed methodologies.     

Choice of generalized linear mixed models using predictive crossvalidation

The choice of generalized linear mixed models is difficult, because it involves the selection of both fixed and random effects. Classical criteria like Akaike’s information criterion (AIC) are often not suitable for the latter task, and others which are useful in linear mixed models are difficult to extend to the generalized case, especially for overdispersed data. A predictive leave-one-out crossvalidation approach is suggested that can be applied for choosing both fixed and random effects, even in models with overdispersion, and is based on proper scoring rules. An attractive feature of this approach is the fact that the model has to be fitted just once to the data set, which makes computations fast and convenient. As the calculation of the leave-one-out predictive distribution is not possible analytically, it is shown how an iteratively weighted least squares algorithm combined with some analytic approximations can be used for this task. A simulation study and two applications of the methodology to binary and count data are provided, as well as comparisons with two other methods.     

Impact of non-normal random effects on inference by multiple imputation: A simulation assessment

Multivariate extensions of well-known linear mixed-effects models have been increasingly utilized in inference by multiple imputation in the analysis of multilevel incomplete data. The normality assumption for the underlying error terms and random effects plays a crucial role in simulating the posterior predictive distribution from which the multiple imputations are drawn. The plausibility of this normality assumption on the subject-specific random effects is assessed. Specifically, the performance of multiple imputation created under a multivariate linear mixed-effects model is investigated on a diverse set of incomplete data sets simulated under varying distributional characteristics. Under moderate amounts of missing data, the simulation study confirms that the underlying model leads to a well-calibrated procedure with negligible biases and actual coverage rates close to nominal rates in estimates of the regression coefficients. Estimation quality of the random-effect variance and association measures, however, are negatively affected from both the misspecification of the random-effect distribution and number of incompletely-observed variables. Some of the adverse impacts include lower coverage rates and increased biases.     

Practical estimation of high dimensional stochastic differential mixed-effects models

Stochastic differential equations (SDEs) are established tools for modeling physical phenomena whose dynamics are affected by random noise. By estimating parameters of an SDE, intrinsic randomness of a system around its drift can be identified and separated from the drift itself. When it is of interest to model dynamics within a given population, i.e. to model simultaneously the performance of several experiments or subjects, mixed-effects modelling allows for the distinction of between and within experiment variability. A framework for modeling dynamics within a population using SDEs is proposed, representing simultaneously several sources of variation: variability between experiments using a mixed-effects approach and stochasticity in the individual dynamics, using SDEs. These stochastic differential mixed-effects models have applications in e.g. pharmacokinetics/pharmacodynamics and biomedical modelling. A parameter estimation method is proposed and computational guidelines for an efficient implementation are given. Finally the method is evaluated using simulations from standard models like the two-dimensional Ornstein-Uhlenbeck (OU) and the square root models.     

Robust concentration graph model selection

Concentration graph models are an attractive tool to explore the conditional independence structure in a multivariate normal distribution. In applications, in absence of a priori knowledge, it is possible to select the graph underlying a set of data through an appropriate model selection procedure. The recently proposed procedure, SINful, is appealing but sensitive to outliers, as it utilizes the sample estimator of the covariance matrix. A method to make the SINful procedure robust with respect to the presence of outlying observations, is proposed. This is based on the minimum covariance determinant (MCD) estimator for the variance-covariance matrix. A simulation study shows the advantages of this method.     

Estimation in the probit normal model for binary outcomes using the SAEM algorithm

Generalized linear mixed models (GLMM) form a very general class of random effects models for discrete and continuous responses in the exponential family. They are useful in a variety of applications. The traditional likelihood approach for GLMM usually involves high dimensional integrations which are computationally intensive. In this work, we investigate the case of binary outcomes analyzed under a two stage probit normal model with random effects. First, it is shown how ML estimates of the fixed effects and variance components can be computed using a stochastic approximation of the EM algorithm (SAEM). The SAEM algorithm can be applied directly, or in conjunction with a parameter expansion version of EM to speed up the convergence. A procedure is also proposed to obtain REML estimates of variance components and REML-based estimates of fixed effects. Finally an application to a real data set involving a clinical trial is presented, in which these techniques are compared to other procedures (penalized quasi-likelihood, maximum likelihood, Bayesian inference) already available in classical softwares (SAS Glimmix, SAS Nlmixed, WinBUGS), as well as to a Monte Carlo EM (MCEM) algorithm.     

Identification of commutative covariance structures by successive testing of statistical hypotheses

For the multidimensional stochastic systems obeying the regression models with unknown covariances of disturbances, consideration was given to the choice of a covariance model and estimation of its parameters. The invariant behavior of the regression model with the covariance matrix of a special structure was studied. In the problem of identifying the structure of a set of feasible covariance matrices, a procedure of successive testing of hypotheses was proposed. The unbiased and invariant uniformly optimal estimates of the parameters of the observation-based model were determined. The problem of identifying the model of covariances in experiment design with random factors was considered as an example.Translated from Avtomatika i Telemekhanika, No. 3, 2005, pp. 48–64.Original Russian Text Copyright © 2005 by Sysoev, Shaikin.This paper was recommended for publication by V.A. Lototskii, a member of the Editorial Board     

A Bayesian regression model for multivariate functional data

In this paper we present a model for the analysis of multivariate functional data with unequally spaced observation times that may differ among subjects. Our method is formulated as a Bayesian mixed-effects model in which the fixed part corresponds to the mean functions, and the random part corresponds to individual deviations from these mean functions. Covariates can be incorporated into both the fixed and the random effects. The random error term of the model is assumed to follow a multivariate Ornstein–Uhlenbeck process. For each of the response variables, both the mean and the subject-specific deviations are estimated via low-rank cubic splines using radial basis functions. Inference is performed via Markov chain Monte Carlo methods.     

Smooth-CAR mixed models for spatial count data

Penalized splines (P-splines) and individual random effects are used for the analysis of spatial count data. P-splines are represented as mixed models to give a unified approach to the model estimation procedure. First, a model where the spatial variation is modelled by a two-dimensional P-spline at the centroids of the areas or regions is considered. In addition, individual area-effects are incorporated as random effects to account for individual variation among regions. Finally, the model is extended by considering a conditional autoregressive (CAR) structure for the random effects, these are the so called “Smooth-CAR” models, with the aim of separating the large-scale geographical trend, and local spatial correlation. The methodology proposed is applied to the analysis of lip cancer incidence rates in Scotland.     

Discrete fuzzy covariance control for specified decay rate

This paper addresses the covariance control problem with decay rate for a class of nonlinear discrete stochastic systems using the Takagi-Sugeno (T-S) fuzzy models. A methodology is developed to find the discrete fuzzy controllers for achieving individual state variance constraints of discrete T-S fuzzy stochastic models. The approach developed in this paper is based on the concept of parallel distributed compensation (PDC) and covariance control. For each rule of the discrete T-S fuzzy model, it shows how to parameterize the static linear state feedback control gains to achieve a common covariance matrix and decay rate for each subsystem. Finally, a numerical example is provided to verify the effects of the proposed design method.     

选用改进高斯过程回归模型的碳排放短期预测

针对于采矿过程中以电机为研究对象的碳排放来源的复杂性以及其影响因素的多样性所引起的碳排放短期预测精度不高的问题,结合灰色理论提出一种基于改进高斯过程回归模型的铅锌矿采矿过程碳排放预测方法。对碳排放来源及其影响因素进行分析,用灰色理论进行聚类分析以归并同类因素;根据灰色关联性分析得到主要影响因素;因传统高斯过程回归模型直接选定协方差函数的方式易导致与研究对象的物理过程拟合度不够高的问题,因而提出了一种依据先验知识的协方差函数选择方式,将四种常用协方差函数建模的训练结果作为反馈,结合极大似然估计法、最小二乘法和蒙特卡洛法参数估计的对比结果得到与研究对象拟合度最高即预测误差最小的协方差函数,进而得到预测效果最好的改进模型。经实验证明,基于该种方法选择协方差函数的模型相较于其他常规预测模型能更精确地预测铅锌矿采矿过程的碳排放量,其预测误差更小。     

Bayesian analysis of multivariate nominal measures using multivariate multinomial probit models

The multinomial probit model has emerged as a useful framework for modeling nominal categorical data, but extending such models to multivariate measures presents computational challenges. Following a Bayesian paradigm, we use a Markov chain Monte Carlo (MCMC) method to analyze multivariate nominal measures through multivariate multinomial probit models. As with a univariate version of the model, identification of model parameters requires restrictions on the covariance matrix of the latent variables that are introduced to define the probit specification. To sample the covariance matrix with restrictions within the MCMC procedure, we use a parameter-extended Metropolis-Hastings algorithm that incorporates artificial variance parameters to transform the problem into a set of simpler tasks including sampling an unrestricted covariance matrix. The parameter-extended algorithm also allows for flexible prior distributions on covariance matrices. The prior specification in the method described here generalizes earlier approaches to analyzing univariate nominal data, and the multivariate correlation structure in the method described here generalizes the autoregressive structure proposed in previous multiperiod multinomial probit models. Our methodology is illustrated through a simulated example and an application to a cancer-control study aiming to achieve early detection of breast cancer.     

Integrated optimization of inventory-distribution systems by random local search and a genetic algorithm

A new model and its solution procedure for the commodity distribution system consisting of distribution centers and consumer points are discussed. Demand is assumed to be a random variable that obeys a known, stationary probability distribution. An integrated optimization model is built where both the order-up-to-R policy, which is one of the typical inventory policies for periodic review models, and the transportation problem are considered simultaneously. The assignment of consumer points to distribution centers is not fixed. The problem is to determine the target inventory and the transportation quantity in order to minimize the expectation of the sum of inventory related costs and transportation costs. Simulation and linear programming are used to calculate the expected costs, and a random local search method is developed in order to determine the optimum target inventory. A genetic algorithm is also tested and compared with the proposed random local search method. The model and effectiveness of the proposed solution procedure are clarified by computational experiments.     

Likelihood and Pseudo-likelihood Methods for Semiparametric Joint Models for a Primary Endpoint and Longitudinal Data

Inference on the association between a primary endpoint and features of longitudinal profiles of a continuous response is of central interest in medical and public health research. Joint models that represent the association through shared dependence of the primary and longitudinal data on random effects are increasingly popular; however, existing inferential methods may be inefficient or sensitive to assumptions on the random effects distribution. We consider a semiparametric joint model that makes only mild assumptions on this distribution and develop likelihood-based inference on the association and distribution, which offers improved performance relative to existing methods that is insensitive to the true random effects distribution. Moreover, the estimated distribution can reveal interesting population features, as we demonstrate for a study of the association between longitudinal hormone levels and bone status in peri-menopausal women.     

On Bayesian estimation and model comparison of an integrated structural equation model

In this paper, we introduce a Bayesian approach to the estimation and model comparison of an integrated two-level nonlinear structural equation model with mixed continuous, dichotomous, and ordered categorical data that may be missing at random. This general model can accommodate nonlinearities of latent variables and the effects of fixed covariates on measurement and structural equations in within-groups and between-groups models. A sampling-based algorithm that combines the Gibbs sampler and the Metropolis-Hastings algorithm is proposed for posterior simulation. A procedure that utilizes path sampling is implemented to compute the Bayes factor for model comparison under the framework of the proposed integrated model. Empirical performances of Bayesian methodologies are illustrated via analysis of a real example.     

RE-EM trees: a data mining approach for longitudinal and clustered data

Longitudinal data refer to the situation where repeated observations are available for each sampled object. Clustered data, where observations are nested in a hierarchical structure within objects (without time necessarily being involved) represent a similar type of situation. Methodologies that take this structure into account allow for the possibilities of systematic differences between objects that are not related to attributes and autocorrelation within objects across time periods. A standard methodology in the statistics literature for this type of data is the mixed effects model, where these differences between objects are represented by so-called “random effects” that are estimated from the data (population-level relationships are termed “fixed effects,” together resulting in a mixed effects model). This paper presents a methodology that combines the structure of mixed effects models for longitudinal and clustered data with the flexibility of tree-based estimation methods. We apply the resulting estimation method, called the RE-EM tree, to pricing in online transactions, showing that the RE-EM tree is less sensitive to parametric assumptions and provides improved predictive power compared to linear models with random effects and regression trees without random effects. We also apply it to a smaller data set examining accident fatalities, and show that the RE-EM tree strongly outperforms a tree without random effects while performing comparably to a linear model with random effects. We also perform extensive simulation experiments to show that the estimator improves predictive performance relative to regression trees without random effects and is comparable or superior to using linear models with random effects in more general situations.