Joint analysis of mixed Poisson and continuous longitudinal data with nonignorable missing values

Regression models are proposed for joint analysis of Poisson and continuous longitudinal data with nonignorable missing values under fully parametric framework. Our primary interest is to evaluate the influence of the covariates on both Poisson and continuous responses. First, we form the full likelihood with complete data using the multivariate Poisson model and conditional multivariate normal distribution and then construct an ECM algorithm to find the maximum likelihood estimates of the model parameters. Then, under the assumption that the missingness mechanisms for the two responses are independent but nonignorable, namely, dependent on both observed and missing data of the two responses, we choose the logit model for the missingness mechanisms and selection model for the full likelihood. Also, we build two implementations of the Monte Carlo EM algorithm for estimating the parameters in the model. Wald test is employed to test the significance of covariates. Finally, we present the results of the Monte Carlo simulation to evaluate the performance of the proposed methodology and an application to the interstitial cystitis data base (ICDB) cohort study. To the best of our knowledge, our model is the first parametric model for joint analysis of Poisson and continuous longitudinal data with nonignorable missing value.     

On Bayesian estimation and model comparison of an integrated structural equation model

In this paper, we introduce a Bayesian approach to the estimation and model comparison of an integrated two-level nonlinear structural equation model with mixed continuous, dichotomous, and ordered categorical data that may be missing at random. This general model can accommodate nonlinearities of latent variables and the effects of fixed covariates on measurement and structural equations in within-groups and between-groups models. A sampling-based algorithm that combines the Gibbs sampler and the Metropolis-Hastings algorithm is proposed for posterior simulation. A procedure that utilizes path sampling is implemented to compute the Bayes factor for model comparison under the framework of the proposed integrated model. Empirical performances of Bayesian methodologies are illustrated via analysis of a real example.     

Bayesian analysis of semiparametric reproductive dispersion mixed-effects models

Semiparametric reproductive dispersion mixed-effects model (SPRDMM) is an extension of the reproductive dispersion model and the semiparametric mixed model, and it includes many commonly encountered models as its special cases. A Bayesian procedure is developed for analyzing SPRDMMs on the basis of P-spline estimates of nonparametric components. A hybrid algorithm combining the Gibbs sampler and the Metropolis-Hastings algorithm is used to simultaneously obtain the Bayesian estimates of unknown parameters, smoothing function and random effects, as well as their standard error estimates. The Bayes factor for model comparison is employed to select better approximation of the smoothing function via path sampling. Several simulation studies and a real example are used to illustrate the proposed methodologies.     

Mixture analysis of multivariate categorical data with covariates and missing entries

Longitudinal or otherwise correlated categorical variables are typically related to some covariates and exhibit nonignorable correlations of the observed variables. A further complication often consists in missing entries. For analyzing such data, it is proposed to create an extra missing category and to employ latent class analysis which, regarding missing data, can be shown to belong to the family of nonmissing at random models. By treating the complete and the incomplete cases jointly, it becomes possible to estimate the parameters of interest along with additional parameters characterizing the missing mechanism. Data from the Muscatine Coronary Risk Factor Study, where each child was classified obese or not obese at three occasions, serve as an illustrative example. Previous analyses resulted in significant interaction of age and sex for the complete data (N=460), and in a linear increase in the logit of the rate of obesity over time for the incomplete data, with no effect of the covariate sex (N=1014). Reanalyses employing latent class models do not support these findings. The finally accepted two-classes model for the complete data assumes a linear effect of age which is the same for boys and girls. The incomplete data were considered three-categorical (not obese, obese, missing) and resulted in a more complex model only in part supporting the linear age hypothesis.     

Identifiability of extended latent class models with individual covariates

Identifiability for a very flexible family of latent class models introduced recently is examined. These models allow for a conditional association between selected pairs of response variables conditionally on the latent and are based on logistic regression models both for the latent weights and for the conditional distributions of the response variables in terms of subject specific covariates. Generalized logits (global or continuation, which are relevant with ordered categorical responses and involve comparisons of cumulated probabilities) may be used as an alternative to the usual logits of type local which are log-linear. A compact matrix formulation for the Jacobian of the parametrization and a simple algorithm for checking local identifiability numerically is described. A few examples involving causal inference are examined.     

Mixture Poisson regression models for heterogeneous count data based on latent and fuzzy class analysis

In this paper we propose a new approach, called a fuzzy class model for Poisson regression, in the analysis of heterogeneous count data. On the basis of fuzzy set concept and fuzzy classification maximum likelihood (FCML) procedures we create an FCML algorithm for fuzzy class Poisson regression models. Traditionally, the EM algorithm had been used for latent class regression models. Thus, the accuracy and effectiveness of EM and FCML algorithms for estimating the parameters are compared. The results show that the proposed FCML algorithm presents better accuracy and effectiveness and can be used as another good tool to regression analysis for heterogeneous count data.This work was supported in part by the National Science Council of Taiwan under Grant NSC-89-2213-E-033-007.     

A latent class selection model for nonignorably missing data

When we have data with missing values, the assumption that data are missing at random is very convenient. It is, however, sometimes questionable because some of the missing values could be strongly related to the underlying true values. We introduce methods for nonignorable multivariate missing data, which assume that missingness is related to the variables in question, and to the additional covariates, through a latent variable measured by the missingness indicators. The methodology developed here is useful for investigating the sensitivity of one’s estimates to untestable assumptions about the missing-data mechanism. A simulation study and data analysis are conducted to evaluate the performance of the proposed method and to compare to that of MAR-based alternatives.     

Nonignorable dropout models for longitudinal binary data with random effects: An application of Monte Carlo approximation through the Gibbs output

The analysis of longitudinal data with nonignorable dropout remains an active area in biostatistics research. Nonignorable dropout (ND) refers to the type of dropout when the probability of dropout depends on the missing observations at or after the time of dropout. Failure to account for such dependence may result in biased inference. Motivated by a methadone clinic data of longitudinal binary observations with dropouts, we propose a conditional first order autoregressive (AR1) logit model for the outcome measurements. The model is further extended to incorporate random effects in order to account for the population heterogeneity and intra-cluster correlation. The purposed models account for the dropout mechanism by a separate logit model in some covariates and missing outcomes for the binary dropout indicators. For model implementation, we proposed a likelihood approach through Monte Carlo approximation to the Gibbs output that evaluates the complicated likelihood function for the random effect ND model without tear. Finally simulation studies are performed to evaluate the biases on the parameter estimates of the outcome model for different dropout mechanisms.     

A computationally efficient method for nonlinear mixed-effects models with nonignorable missing data in time-varying covariates

Nonlinear mixed-effects (NLME) models are widely used for longitudinal data analyses. Time-dependent covariates are often introduced to partially explain inter-individual variation. These covariates often have missing data, and the missingness may be nonignorable. Likelihood inference for NLME models with nonignorable missing data in time-varying covariates can be computationally very intensive and may even offer computational difficulties such as nonconvergence. We propose a computationally very efficient method for approximate likelihood inference. The method is illustrated using a real data example.     

A new approach to formulate fuzzy regression models

A fuzzy regression model is developed to construct the relationship between the response and explanatory variables in fuzzy environments. To enhance explanatory power and take into account the uncertainty of the formulated model and parameters, a new operator, called the fuzzy product core (FPC), is proposed for the formulation processes to establish fuzzy regression models with fuzzy parameters using fuzzy observations that include fuzzy response and explanatory variables. In addition, the sign of parameters can be determined in the model-building processes. Compared to existing approaches, the proposed approach reduces the amount of unnecessary or unimportant information arising from fuzzy observations and determines the sign of parameters in the models to increase model performance. This improves the weakness of the relevant approaches in which the parameters in the models are fuzzy and must be predetermined in the formulation processes. The proposed approach outperforms existing models in terms of distance, mean similarity, and credibility measures, even when crisp explanatory variables are used.     

嵌套删失数据期望最大化的高斯混合聚类算法

针对聚类问题中的非随机性缺失数据,本文基于高斯混合聚类模型,分析了删失型数据期望最大化算法的有效性,并揭示了删失数据似然函数对模型算法的作用机制.从赤池弘次信息准则、信息散度等指标,比较了所提出方法与标准的期望最大化算法的优劣性.通过删失数据划分及指示变量,推导了聚类模型参数后验概率及似然函数,调整了参数截尾正态函数的...     

Favorability functions based on kernel density estimation for logistic models: A case study

Susceptibility or hazard models are often established by means of logistic regression techniques in order to describe the effect of a group of explanatory variables on the probability of a dichotomous or binary response. Since the available variables do not always meet the assumptions of logit-linearity of the logistic regression, a modified approach is proposed. Firstly a favorability function associated with each explanatory variable based on the conditional probability measures is introduced. Next, a simple transformation based on the empirical probability function for non-continuous variables is suggested, while nonparametric kernel estimation is considered for continuous ones. The favorability-based transformations lead to new explanatory variables for the logistic regression model. The performance of the method is evaluated using simulated data. In addition, a real case-study is presented, in which a GIS-based landslides susceptibility model is carried out.     

基于粒子群算法的Logistic回归模型参数估计

针对Logistic回归模型中的参数估计计算复杂难题,提出一种基于粒子群优化算法（PSO）的估计方法。以最大似然准则作为粒子群优化算法的适应度函数,建立了Logistic回归模型中的参数估算模型。数值仿真分析表明,粒子群优化算法可以更精确地计算出相关参数。     

Latent-space variational bayes

Variational Bayesian Expectation-Maximization (VBEM), an approximate inference method for probabilistic models based on factorizing over latent variables and model parameters, has been a standard technique for practical Bayesian inference. In this paper, we introduce a more general approximate inference framework for conjugate-exponential family models, which we call Latent-Space Variational Bayes (LSVB). In this approach, we integrate out model parameters in an exact way, leaving only the latent variables. It can be shown that the LSVB approach gives better estimates of the model evidence as well as the distribution over latent variables than the VBEM approach, but in practice, the distribution over latent variables has to be approximated. As a practical implementation, we present a First-order LSVB (FoLSVB) algorithm to approximate this distribution over latent variables. From this approximate distribution, one can estimate the model evidence and the posterior over model parameters. The FoLSVB algorithm is directly comparable to the VBEM algorithm and has the same computational complexity. We discuss how LSVB generalizes the recently proposed collapsed variational methods [20] to general conjugate-exponential families. Examples based on mixtures of Gaussians and mixtures of Bernoullis with synthetic and real-world data sets are used to illustrate some advantages of our method over VBEM.     

A novel logistic multi-class supervised classification model based on multi-fractal spectrum parameters for hyperspectral data

A novel logistic multi-class supervised classification model based on multi-fractal spectrum parameters is proposed to avoid the error that is caused by the difference between the real data distribution and the hypothetic Gaussian distribution and avoid the computational burden working in the logistic regression classification directly for hyperspectral data. The multi-fractal spectra and parameters are calculated firstly with training samples along the spectral dimension of hyperspectral data. Secondly, the logistic regression model is employed in our work because the logistic regression classification model is a distribution-free nonlinear model which is based on the conditional probability without the Gaussian distribution assumption of the random variables, and the obtained multi-fractal parameters are applied to establish the multi-class logistic regression classification model. Finally, the Newton–Raphson method is applied to estimate the model parameters via the maximum likelihood algorithm. The classification results of the proposed model are compared with the logistic regression classification model based on an adaptive bands selection method by using the Airborne Visible/Infrared Imaging Spectrometer and airborne Push Hyperspectral Imager data. The results illuminate that the proposed approach achieves better accuracy with lower computational cost simultaneously.     

Approximate conditional inference in mixed-effects models with binary data

The conditional likelihood approach is a sensible choice for a hierarchical logistic regression model or other generalized regression models with binary data. However, its heavy computational burden limits its use, especially for the related mixed-effects model. A modified profile likelihood is used as an accurate approximation to conditional likelihood, and then the use of two methods for inferences for the hierarchical generalized regression models with mixed effects is proposed. One is based on a hierarchical likelihood and Laplace approximation method, and the other is based on a Markov chain Monte Carlo EM algorithm. The methods are applied to a meta-analysis model for trend estimation and the model for multi-arm trials. A simulation study is conducted to illustrate the performance of the proposed methods.     

Jackknife and bootstrap methods in the identification of dynamic models

A new criterion based on a Jackknife or a Bootstrap statistic is proposed for identifying non-parsimonious dynamic models (FIR, ARX). It is applicable for selecting the number of components in latent variable regression methods or the constraining parameter in regularized least squares regression methods. These meta parameters are used to overcome ill-conditioning caused by model over-parameterization, when fitted using prediction error or least squares methods. In all cases studied, using PLS for parameter estimation, the proposed criterion led to the selection of better models, in the mean square error sense, than when selected via cross-validation. The methodology also provides approximate confidence intervals for the model parameters and the step and impulse response of the system.     

Latent log-linear models for handwritten digit classification

We present latent log-linear models, an extension of log-linear models incorporating latent variables, and we propose two applications thereof: log-linear mixture models and image deformation-aware log-linear models. The resulting models are fully discriminative, can be trained efficiently, and the model complexity can be controlled. Log-linear mixture models offer additional flexibility within the log-linear modeling framework. Unlike previous approaches, the image deformation-aware model directly considers image deformations and allows for a discriminative training of the deformation parameters. Both are trained using alternating optimization. For certain variants, convergence to a stationary point is guaranteed and, in practice, even variants without this guarantee converge and find models that perform well. We tune the methods on the USPS data set and evaluate on the MNIST data set, demonstrating the generalization capabilities of our proposed models. Our models, although using significantly fewer parameters, are able to obtain competitive results with models proposed in the literature.     

Mixed-membership naive Bayes models

In recent years, mixture models have found widespread usage in discovering latent cluster structure from data. A popular special case of finite mixture models is the family of naive Bayes (NB) models, where the probability of a feature vector factorizes over the features for any given component of the mixture. Despite their popularity, naive Bayes models do not allow data points to belong to different component clusters with varying degrees, i.e., mixed memberships, which puts a restriction on their modeling ability. In this paper, we propose mixed-membership naive Bayes (MMNB) models. On one hand, MMNB can be viewed as a generalization of NB by putting a Dirichlet prior on top to allow mixed memberships. On the other hand, MMNB can also be viewed as a generalization of latent Dirichlet allocation (LDA) with the ability to handle heterogeneous feature vectors with different types of features, e.g., real, categorical, etc.. We propose two variational inference algorithms to learn MMNB models. The first one is based on ideas originally used in LDA, and the second one uses substantially fewer variational parameters, leading to a significantly faster algorithm. Further, we extend MMNB/LDA to discriminative mixed-membership models for classification by suitably combining MMNB/LDA with multi-class logistic regression. The efficacy of the proposed mixed-membership models is demonstrated by extensive experiments on several datasets, including UCI benchmarks, recommendation systems, and text datasets.     

一种任意分布下的隐变量因果结构学习算法

因果发现旨在通过观测数据挖掘变量间的因果关系,在实际应用中需要从观测数据中学习隐变量间的因果结构。现有方法主要利用观测变量间的协方差信息（如四分体约束）或引入非高斯假设（如三分体约束）来解决线性因果模型下的隐变量结构学习问题,但大多限定于分布明确的情况,而实际应用环境往往并不满足这种假设。给出任意分布下隐变量结构的识别性证明,指出在没有混淆因子影响的情况下,两个隐变量的因果方向可识别所需要的最小条件是仅需要其中一个隐变量的噪声服从非高斯分布。在此基础上,针对线性隐变量模型提出一种在任意分布下学习隐变量因果结构的算法,先利用四分体约束方法学习得到隐变量骨架图,再通过枚举骨架图的等价类并测量每一个等价类中的三分体约束来学习因果方向,同时将非高斯约束放宽到尽可能最小的变量子集,从而扩展线性隐变量模型的应用范围。实验结果表明,与MIMBuild和三分体约束方法相比,该算法得到了最佳的F1值,能够在任意分布下学习更多的隐变量因果结构信息,且具有更强的鲁棒性。