A novel method for testing normality in a mixed model of a nested classification

Normality is one of the most common assumptions made in the development of statistical models such as the fixed effect model and the random effect model. White and MacDonald [1980. Some large-sample tests for normality in the linear regression model. JASA 75, 16-18] and Bonett and Woodward [1990. Testing residual normality in the ANOVA model. J. Appl. Statist. 17, 383-387] showed that many tests of normality perform well when applied to the residuals of a fixed effect model. The elements of the error vector are not independent in random effects models and standard tests of normality are not expected to perform properly when applied to the residuals of a random effects model.In this paper, we propose a transformation method to convert the correlated error vector into an uncorrelated vector. Moreover, under the normality assumption, the uncorrelated vector becomes an independent vector. Thus, all the existing methods can then be implemented. Monte-Carlo simulations are used to evaluate the feasibility of the transformation. Results show that this transformation method can preserve the Type I error and provide greater powers under most alternatives.     

Generalized linear mixed model with a penalized Gaussian mixture as a random effects distribution

Generalized linear mixed models are popular for regressing a discrete response when there is clustering, e.g. in longitudinal studies or in hierarchical data structures. It is standard to assume that the random effects have a normal distribution. Recently, it has been examined whether wrongly assuming a normal distribution for the random effects is important for the estimation of the fixed effects parameters. While it has been shown that misspecifying the distribution of the random effects has a minor effect in the context of linear mixed models, the conclusion for generalized mixed models is less clear. Some studies report a minor impact, while others report that the assumption of normality really matters especially when the variance of the random effect is relatively high. Since it is unclear whether the normality assumption is truly satisfied in practice, it is important that generalized mixed models are available which relax the normality assumption. A replacement of the normal distribution with a mixture of Gaussian distributions specified on a grid whereby only the weights of the mixture components are estimated using a penalized approach ensuring a smooth distribution for the random effects is proposed. The parameters of the model are estimated in a Bayesian context using MCMC techniques. The usefulness of the approach is illustrated on two longitudinal studies using R-functions.     

Generalized linear models with clustered data: Fixed and random effects models

The statistical analysis of mixed effects models for binary and count data is investigated. In the statistical computing environment R, there are a few packages that estimate models of this kind. The package lme4 is a de facto standard for mixed effects models. The package glmmML allows non-normal distributions in the specification of random intercepts. It also allows for the estimation of a fixed effects model, assuming that all cluster intercepts are distinct fixed parameters; moreover, a bootstrapping technique is implemented to replace asymptotic analysis. The random intercepts model is fitted using a maximum likelihood estimator with adaptive Gauss-Hermite and Laplace quadrature approximations of the likelihood function. The fixed effects model is fitted through a profiling approach, which is necessary when the number of clusters is large. In a simulation study, the two approaches are compared. The fixed effects model has severe bias when the mixed effects variance is positive and the number of clusters is large.     

Robust steady-state tracking

This paper addresses the robust performance problem when the performance measure is the “steady-state” value of an error signal. Necessary and sufficient conditions are derived for robust steady-state tracking of fixed inputs in the presence of structured time-varying uncertainty are derived. These conditions are easily computable and fit well with existing conditions on stability robustness and performance robustness when the performance measure is the level of disturbance rejection. Using these conditions, it is shown that time-varying perturbations of a nominal linear time-invariant plant can result in large steady-state tracking errors to fixed inputs even if the nominal plant and controller give zero steady-state tracking errors. The derived expressions for the worst-case steady-state tracking error give insight into how the time variation in the plant affects tracking errors and suggest that certain transfer function norms should be minimized to reduce the effect of these perturbations on the steady-state value of error signals     

Flexible modelling of random effects in linear mixed models—A Bayesian approach

Flexible modelling of random effects in linear mixed models has attracted some attention recently. In this paper, we propose the use of finite Gaussian mixtures as in Verbeke and Lesaffre [A linear mixed model with heterogeneity in the random-effects population, J. Amu. Statist. Assoc. 91, 217-221]. We adopt a fully Bayesian hierarchical framework that allows simultaneous estimation of the number of mixture components together with other model parameters. The technique employed is the Reversible Jump MCMC algorithm (Richardson and Green [On Bayesian Analysis of Mixtures with an Unknown Number of Components (with discussion). J. Roy. Statist. Soc. Ser. B 59, 731-792]). This approach has the advantage of producing a direct comparison of different mixture models through posterior probabilities from a single run of the MCMC algorithm. Moreover, the Bayesian setting allows us to integrate over different mixture models to obtain a more robust density estimate of the random effects. We focus on linear mixed models with a random intercept and a random slope. Numerical results on simulated data sets and a real data set are provided to demonstrate the usefulness of the proposed method.     

A Bayesian regression model for multivariate functional data

In this paper we present a model for the analysis of multivariate functional data with unequally spaced observation times that may differ among subjects. Our method is formulated as a Bayesian mixed-effects model in which the fixed part corresponds to the mean functions, and the random part corresponds to individual deviations from these mean functions. Covariates can be incorporated into both the fixed and the random effects. The random error term of the model is assumed to follow a multivariate Ornstein–Uhlenbeck process. For each of the response variables, both the mean and the subject-specific deviations are estimated via low-rank cubic splines using radial basis functions. Inference is performed via Markov chain Monte Carlo methods.     

Multilevel cumulative logistic regression model with random effects: Application to British social attitudes panel survey data

A multilevel model for ordinal data in generalized linear mixed models (GLMM) framework is developed to account for the inherent dependencies among observations within clusters. Motivated by a data set from the British Social Attitudes Panel Survey (BSAPS), the random district effects and respondent effects are incorporated into the linear predictor to accommodate the nested clusterings. The fixed (random) effects are estimated (predicted) by maximizing the penalized quasi likelihood (PQL) function, whereas the variance component parameters are obtained via the restricted maximum likelihood (REML) estimation method. The model is employed to analyze the BSAPS data. Simulation studies are conducted to assess the performance of estimators.     

Error handling strategies in multiphase inverse modeling

Parameter estimation by inverse modeling involves the repeated evaluation of a function of residuals. These residuals represent both errors in the model and errors in the data. In practical applications of inverse modeling of multiphase flow and transport, the error structure of the final residuals often significantly deviates from the statistical assumptions that underlie standard maximum likelihood estimation using the least-squares method. Large random or systematic errors are likely to lead to convergence problems, biased parameter estimates, misleading uncertainty measures, or poor predictive capabilities of the calibrated model. The multiphase inverse modeling code iTOUGH2 supports strategies that identify and mitigate the impact of systematic or non-normal error structures. We discuss these approaches and provide an overview of the error handling features implemented in iTOUGH2.     

Adaptive control of discrete-time systems using multiple models

The adaptive control of a linear time-invariant discrete-time system using multiple models is considered in this paper. Both the deterministic (noise free) case and the stochastic case when random disturbances are present are discussed. Based on the prediction errors of a finite number of fixed and adaptive identification models, a procedure is outlined for switching between a finite number of controllers to improve performance. The principal contributions of the paper are the proof of global stability of the overall system and the convergence of the tracking error signal to zero in the deterministic case and the proof of convergence of the minimum variance control problem. Computer simulation results are included to complement the theoretical results     

A variance shift model for detection of outliers in the linear mixed model

A variance shift outlier model (VSOM), previously used for detecting outliers in the linear model, is extended to the variance components model. This VSOM accommodates outliers as observations with inflated variance, with the status of the ith observation as an outlier indicated by the size of the associated shift in the variance. Likelihood ratio and score test statistics are assessed as objective measures for determining whether the ith observation has inflated variance and is therefore an outlier. It is shown that standard asymptotic distributions do not apply to these tests for a VSOM, and a modified distribution is proposed. A parametric bootstrap procedure is proposed to account for multiple testing. The VSOM framework is extended to account for outliers in random effects and is shown to have an advantage over case-deletion approaches. A simulation study is presented to verify the performance of the proposed tests. Challenges associated with computation and extensions of the VSOM to the general linear mixed model with correlated errors are discussed.     

An algorithm for robust linear estimation with grouped data

An algorithm which is valid to estimate the parameters of linear models under several robust conditions is presented. With respect to the robust conditions, firstly, the dependent variables may be either non-grouped or grouped. Secondly, the distribution of the errors may vary within the wide class of the strongly unimodal distributions, either symmetrical or non-symmetrical. Finally, the variance of the errors is unknown. Under these circumstances the algorithm is not only capable of estimating the parameters (slopes and error variance) of the linear model, but also the asymptotic covariance matrix of the linear parameters. This opens the possibility of making inferences in terms of either multiple confidence regions or hypothesis testing.     

An algorithm for robust linear estimation with grouped data

An algorithm which is valid to estimate the parameters of linear models under several robust conditions is presented. With respect to the robust conditions, firstly, the dependent variables may be either non-grouped or grouped. Secondly, the distribution of the errors may vary within the wide class of the strongly unimodal distributions, either symmetrical or non-symmetrical. Finally, the variance of the errors is unknown. Under these circumstances the algorithm is not only capable of estimating the parameters (slopes and error variance) of the linear model, but also the asymptotic covariance matrix of the linear parameters. This opens the possibility of making inferences in terms of either multiple confidence regions or hypothesis testing.     

Estimation in the probit normal model for binary outcomes using the SAEM algorithm

Generalized linear mixed models (GLMM) form a very general class of random effects models for discrete and continuous responses in the exponential family. They are useful in a variety of applications. The traditional likelihood approach for GLMM usually involves high dimensional integrations which are computationally intensive. In this work, we investigate the case of binary outcomes analyzed under a two stage probit normal model with random effects. First, it is shown how ML estimates of the fixed effects and variance components can be computed using a stochastic approximation of the EM algorithm (SAEM). The SAEM algorithm can be applied directly, or in conjunction with a parameter expansion version of EM to speed up the convergence. A procedure is also proposed to obtain REML estimates of variance components and REML-based estimates of fixed effects. Finally an application to a real data set involving a clinical trial is presented, in which these techniques are compared to other procedures (penalized quasi-likelihood, maximum likelihood, Bayesian inference) already available in classical softwares (SAS Glimmix, SAS Nlmixed, WinBUGS), as well as to a Monte Carlo EM (MCEM) algorithm.     

Information methods for model selection in linear mixed effects models with application to HCV data

In this paper, we derive a small sample Akaike information criterion, based on the maximized loglikelihood, and a small sample information criterion based on the maximized restricted loglikelihood in the linear mixed effects model when the covariance matrix of the random effects is known. Small sample corrected information criteria are proposed for a special case of linear mixed effects models, the balanced random-coefficient model, without assuming the random coefficients covariance matrix to be known. A simulation study comparing the derived criteria and several others for model selection in the linear mixed effects models is presented. We illustrate the behavior of the studied information criteria on real data from a study of subjects coinfected with HIV and Hepatitis C virus. Robustness of the criteria, in terms of the error distributed as a mixture of normal distributions, is also studied. Special attention is given to the behavior of the conditional AIC by Vaida and Blanchard (2005). Among the studied criteria, GIC performs best, while cAIC exhibits poor performance. Because of its inferior performance, as demonstrated in this work, we do not recommend its use for model selection in linear mixed effects models.     

Pairwise likelihood for the longitudinal mixed Rasch model

Inference in Generalized linear mixed models with multivariate random effects is often made cumbersome by the high-dimensional intractable integrals involved in the marginal likelihood. An inferential methodology based on the marginal pairwise likelihood approach is proposed. This method belonging to the broad class of composite likelihood involves marginal pairs probabilities of the responses which has analytical expression for the probit version of the model, from where we derived those of the logit version. The different results are illustrated with a simulation study and with an analysis of a real data from health-related quality of life.     

Multiobjective filter design for uncertain stochastic time-delay systems

This note addresses the problem of robust multiobjective filtering for discrete time-delay systems with mixed stochastic and deterministic uncertainties, in addition to unmodeled nonlinearities. A procedure is developed for the design of linear and exponentially stable filters with a bounded error variance, exponential rate of decay, and robust performance for the error system.     

Evolution of regulation errors in the diffusion layer in electrodeposition process control

A random field induced by infinitely many noises emerges in numerous processes, such as a mixed boundary electrochemical deposition process, which is determined by the main reaction and several side reactions. Since the process is controlled on one boundary without knowledge of the side reactions, a model error emerges and the regulation error trails. The model considered in this paper allows for general settings and is defined for white noise and colour noise. The white noise is responsible for unpredictable model errors and the colour noise for systematic errors. A relatively simple boundary control is applied that, when lifted into the domain, acts as a smooth function. The paper demonstrates that the regulation error can be suppressed effectively even if the correlation of random noises is relatively weak. The regulation error is suppressed by the double effects of the Laplacian and noise correlation. The traces of covariance operators are found for several categories of the domain noises and for the boundary noises. A similar trace function in general settings satisfies a Kolmogorov equation in infinitely many state variables. The solution for the Kolmogorov equation is simplified to a form that makes numerical treatment possible. The regulation errors induced by white noise are compared with the errors induced by coloured noise in the numerical analysis.     

Multilevel modeling using spatial processes: Application to the Singapore housing market

Customary spatial modeling with point-referenced data introduces a modeling specification that includes a mean term, a spatial error or random effects term and a pure error term. The spatial random effects are usually modeled through a mean zero spatial process. If the mean term includes an intercept, then the spatial random effects can be interpreted as local spatial adjustments to the intercept. If the mean term is a familiar linear regression then it makes sense to ask whether the regression coefficients are constant or whether they might vary spatially, analogous to the intercept. This has been previously considered and the benefits of the increased flexibility have been demonstrated. The situation with replicates available at spatial locations is considered. This enables the building of the spatial analog of a multilevel model—replicate level covariates to explain the replicate level responses and location level covariates to explain the location level coefficients. The particular motivation for this modeling effort is a data set on condominium sales in Singapore. In this case, the replicates are the sales of condominiums within a building. Unit level features are available to explain the selling price of the unit and building level attributes to explain the coefficients. Anticipating dependence between coefficients, a multivariate spatial process specification is provided. This process is specified through kernel convolutions due to the computational challenges associated with fitting such models to a fairly large data set. There is flexibility in this kernel modeling necessitating model comparison. In particular, roughly 68,000 transactions across 1374 buildings (locations) are analyzed and the results and interpretation for the selected model are presented.     

Robust bandwidth selection in semiparametric partly linear regression models: Monte Carlo study and influential analysis

In this paper, under a semiparametric partly linear regression model with fixed design, we introduce a family of robust procedures to select the bandwidth parameter. The robust plug-in proposal is based on nonparametric robust estimates of the νth derivatives and under mild conditions, it converges to the optimal bandwidth. A robust cross-validation bandwidth is also considered and the performance of the different proposals is compared through a Monte Carlo study. We define an empirical influence measure for data-driven bandwidth selectors and, through it, we study the sensitivity of the data-driven bandwidth selectors. It appears that the robust selector compares favorably to its classical competitor, despite the need to select a pilot bandwidth when considering plug-in bandwidths. Moreover, the plug-in procedure seems to be less sensitive than the cross-validation in particular, when introducing several outliers. When combined with the three-step procedure proposed by Bianco and Boente [2004. Robust estimators in semiparametric partly linear regression models. J. Statist. Plann. Inference 122, 229-252] the robust selectors lead to robust data-driven estimates of both the regression function and the regression parameter.     

A mixed effects least squares support vector machine model for classification of longitudinal data

A mixed effects least squares support vector machine (LS-SVM) classifier is introduced to extend the standard LS-SVM classifier for handling longitudinal data. The mixed effects LS-SVM model contains a random intercept and allows to classify highly unbalanced data, in the sense that there is an unequal number of observations for each case at non-fixed time points. The methodology consists of a regression modeling and a classification step based on the obtained regression estimates. Regression and classification of new cases are performed in a straightforward manner by solving a linear system. It is demonstrated that the methodology can be generalized to deal with multi-class problems and can be extended to incorporate multiple random effects. The technique is illustrated on simulated data sets and real-life problems concerning human growth.