Comparing principal stratification and selection models in parametric causal inference with nonignorable missingness

Two approaches for dealing with “endogenous selection” problems when estimating causal effects are considered. They are principal stratification and selection models. The main goal is to highlight similarities and differences between the two approaches, by investigating the different nature of their parametric hypotheses. The principal stratification approach focuses on information contained in specific subgroups of units. The aim is to produce valid inference conditional on such subgroups, without an a priori extension of the results to the whole population. Selection models, on the contrary, aim at estimating parameters that should be valid for the whole population, as if the data come from random sampling. A simulation study is conducted to show their different performances, with data generating processes coming from either approach. It is also argued that principal stratification is able to suggest alternative identification strategies not always easily translatable into assumptions of a selection model.     

On causal inference in fuzzy cognitive maps

Fuzzy cognitive maps (FCM) is a powerful paradigm for representing human knowledge and causal inference. This paper formally analyzes the causal inference mechanism of FCM. We focus on binary concept states. It is known that given initial conditions, FCM is able to reach only certain states in its state space. We prove that the problem of finding whether a state is reachable in the FCM is nondeterministic polynomial (NP) hard, that we can divide fuzzy cognitive maps containing circles into basic FCM modules. The inference patterns in these basic modules can be studied individually in a hierarchical fashion. This paper also presents a recursive formula for computing FCM's inference patterns in terms of key vertices. The theoretical results presented in this paper provide a feasible and effective framework for the analysis and design of fuzzy cognitive maps in real-world large-scale applications     

G-computation estimation for causal inference with complex longitudinal data

In a companion paper, [Neugebauer, R., van der Laan, M.J., 2006b. Causal effects in longitudinal studies: definition and maximum likelihood estimation. Comput. Stat. Data. Anal., this issue, doi:10.1016/j.csda.2006.06.013], we provided an overview of causal effect definition with marginal structural models (MSMs) in longitudinal studies. A parametric MSM (PMSM) and a non-parametric MSM (NPMSM) approach were described for the representation of causal effects in pooled or stratified analyses of treatment effects on time-dependent outcomes. Maximum likelihood estimation, also referred to as G-computation estimation, was detailed for these causal effects. In this paper, we develop new algorithms for the implementation of the G-computation estimators of both NPMSM and PMSM causal effects. Current algorithms rely on Monte Carlo simulation of all possible treatment-specific outcomes, also referred to as counterfactuals or potential outcomes. This task becomes computationally impracticable (a) in studies with a continuous treatment, and/or (b) in longitudinal studies with long follow-up with or without time-dependent outcomes. The proposed algorithms address this important computing limitation inherent to G-computation estimation in most longitudinal studies. Finally, practical considerations about the proposed algorithms lead to a further generalization of the definition of NPMSM causal effects in order to allow more reliable applications of these methodologies to a broader range of real-life studies. Results are illustrated with two simulation studies.     

Comparing measures of model selection for penalized splines in Cox models

This article presents an application and a simulation study of model fit criteria for selecting the optimal degree of smoothness for penalized splines in Cox models. The criteria considered were the Akaike information criterion, the corrected AIC, two formulations of the Bayesian information criterion, and a generalized cross-validation method. The estimated curves selected by the five methods were compared to each other in a study of rectal cancer mortality in autoworkers. In the stimulation study, we estimated the fit of the penalized spline models in six exposure-response scenarios, using the five model fit criteria. The methods were compared on the basis of a mean squared error score and the power and size of hypothesis tests for any effect and for detecting nonlinearity. All comparisons were made across a range in the total sample size and number of cases.     

Local influence analysis of nonlinear structural equation models with nonignorable missing outcomes from reproductive dispersion models

Nonlinear structural equation models with nonignorable missing outcomes from reproductive dispersion models are proposed to identify the relationship between manifest variables and latent variables in modern educational, medical, social and psychological studies. The nonignorable missing mechanism is specified by a logistic regression model. An EM algorithm is developed to obtain the maximum likelihood estimates of the structural parameters and parameters in the logistic regression model. Assessment of local influence is investigated in nonlinear structural equation models with nonignorable missing outcomes from reproductive dispersion models on the basis of the conditional expectation of the complete-data log-likelihood function. Some local influence diagnostics are obtained via observations of missing data and latent variables that are generated by the Gibbs sampler and Metropolis-Hastings algorithm on the basis of the conformal normal curvature. A simulation study and a real example are used to illustrate the application of the proposed methodologies.     

A computationally efficient method for nonlinear mixed-effects models with nonignorable missing data in time-varying covariates

Nonlinear mixed-effects (NLME) models are widely used for longitudinal data analyses. Time-dependent covariates are often introduced to partially explain inter-individual variation. These covariates often have missing data, and the missingness may be nonignorable. Likelihood inference for NLME models with nonignorable missing data in time-varying covariates can be computationally very intensive and may even offer computational difficulties such as nonconvergence. We propose a computationally very efficient method for approximate likelihood inference. The method is illustrated using a real data example.     

Improving probabilistic inference in graphical models with determinism and cycles

Many important real-world applications of machine learning, statistical physics, constraint programming and information theory can be formulated using graphical models that involve determinism and cycles. Accurate and efficient inference and training of such graphical models remains a key challenge. Markov logic networks (MLNs) have recently emerged as a popular framework for expressing a number of problems which exhibit these properties. While loopy belief propagation (LBP) can be an effective solution in some cases; unfortunately, when both determinism and cycles are present, LBP frequently fails to converge or converges to inaccurate results. As such, sampling based algorithms have been found to be more effective and are more popular for general inference tasks in MLNs. In this paper, we introduce Generalized arc-consistency Expectation Maximization Message-Passing (GEM-MP), a novel message-passing approach to inference in an extended factor graph that combines constraint programming techniques with variational methods. We focus our experiments on Markov logic and Ising models but the method is applicable to graphical models in general. In contrast to LBP, GEM-MP formulates the message-passing structure as steps of variational expectation maximization. Moreover, in the algorithm we leverage the local structures in the factor graph by using generalized arc consistency when performing a variational mean-field approximation. Thus each such update increases a lower bound on the model evidence. Our experiments on Ising grids, entity resolution and link prediction problems demonstrate the accuracy and convergence of GEM-MP over existing state-of-the-art inference algorithms such as MC-SAT, LBP, and Gibbs sampling, as well as convergent message passing algorithms such as the concave–convex procedure, residual BP, and the L2-convex method.     

A theoretical analysis based on causal inference and single-instance learning

Applied Intelligence - Although using single-instance learning methods to solve multi-instance problems has achieved excellent performance in many tasks, the reasons for this success still lack a...     

GARCH dependence in extreme value models with Bayesian inference

Extreme value methods are widely used in financial applications such as risk analysis, forecasting and pricing models. One of the challenges with their application in finance is accounting for the temporal dependence between the observations, for example the stylised fact that financial time series exhibit volatility clustering. Various approaches have been proposed to capture the dependence. Commonly a two-stage approach is taken, where the volatility dependence is removed using a volatility model like a GARCH (or one of its many incarnations) followed by application of standard extreme value models to the assumed independent residual innovations.This study examines an alternative one stage approach, which makes parameter estimation and accounting for the associated uncertainties more straightforward than the two-stage approach. The location and scale parameters of the extreme value distribution are defined to follow a conditional autoregressive heteroscedasticity process. Essentially, the model implements GARCH volatility via the extreme value model parameters. Bayesian inference is used and implemented via Markov chain Monte Carlo, to permit all sources of uncertainty to be accounted for. The model is applied to both simulated and empirical data to demonstrate performance in extrapolating the extreme quantiles and quantifying the associated uncertainty.     

Approximate conditional inference in mixed-effects models with binary data

The conditional likelihood approach is a sensible choice for a hierarchical logistic regression model or other generalized regression models with binary data. However, its heavy computational burden limits its use, especially for the related mixed-effects model. A modified profile likelihood is used as an accurate approximation to conditional likelihood, and then the use of two methods for inferences for the hierarchical generalized regression models with mixed effects is proposed. One is based on a hierarchical likelihood and Laplace approximation method, and the other is based on a Markov chain Monte Carlo EM algorithm. The methods are applied to a meta-analysis model for trend estimation and the model for multi-arm trials. A simulation study is conducted to illustrate the performance of the proposed methods.     

Dimension selection for feature selection and dimension reduction with principal and independent component analysis

This letter is concerned with the problem of selecting the best or most informative dimension for dimension reduction and feature extraction in high-dimensional data. The dimension of the data is reduced by principal component analysis; subsequent application of independent component analysis to the principal component scores determines the most nongaussian directions in the lower-dimensional space. A criterion for choosing the optimal dimension based on bias-adjusted skewness and kurtosis is proposed. This new dimension selector is applied to real data sets and compared to existing methods. Simulation studies for a range of densities show that the proposed method performs well and is more appropriate for nongaussian data than existing methods.     

Estimation and inference in functional mixed-effects models

Functional mixed-effects models are very useful in analyzing functional data. A general functional mixed-effects model that inherits the flexibility of linear mixed-effects models in handling complex designs and correlation structures is considered. A wavelet decomposition approach is used to model both fixed-effects and random-effects in the same functional space, meaning that the population-average curve and the subject-specific curves have the same smoothness property. A linear mixed-effects representation is then obtained that is used for estimation and inference in the general functional mixed-effects model. Adapting recent methodologies in linear mixed-effects and nonparametric regression models, hypothesis testing procedures for both fixed-effects and random-effects are provided. Using classical linear mixed-effects estimation techniques, the linear mixed-effects representation is also used to obtain wavelet-based estimates for both fixed-effects and random-effects in the general functional mixed-effects model. The usefulness of the proposed estimation and hypothesis testing procedures is illustrated by means of a small simulation study and a real-life dataset arising from physiology.     

Comparing three global parametric and local non-parametric models to simulate land use change in diverse areas of the world

This paper compares one global parametric land use change model, the artificial neural network – based Land Transformation Model, with two local non-parametric models: a classification and regression tree and multivariate adaptive regression spline model. We parameterized these three models with identical data from different regions of the world; one region undergoing extensive agricultural expansion (East Africa), another region where forests are re-growing (Muskegon River Watershed in the United States), and a third region where urbanization is prominent (South-Eastern Wisconsin in the United States). Independent training data and testing data were used to train and calibrate each model, respectively. Comparisons of simulated maps from the three kinds of land use change patterns were made using conventional goodness-of-fit metrics in land use change science. The results of temporal and spatial comparison of the data mining models show that the artificial neural network outperformed all other models in a short-time interval (East Africa; 5 years) and for coarse resolution data (East Africa; 1 km); however, the three data mining models obtained similar accuracies in a long-time interval (Muskegon River Watershed; 20 years) and for fine resolution data with large numbers of cells (Muskegon River Watershed; 30 m). Furthermore, the results showed that the probability of agriculture gain was more likely in locations closer to towns and large cities in East Africa, urbanization was more likely in locations closer to roads and urban areas in South-Eastern Wisconsin and the probability of forest gain was more likely in locations closer to the forest and shrub land cover and farther away from roads in Muskegon River Watershed.     

A method for comparing semi-parametric models with parametric models in competing risks analysis

In survival analysis, Cox's regression model is often used to assess the effect of covariates on the time of failure. This semi-parametric model has been extended to the situation where more than one cause of failure is of interest. In this paper, two semi-parametric models for the analysis of competing risks with covariates in the presence of independent random censoring are considered. Particular attention is devoted to the comparison between the two models. A method using a measure derived from the generalized variance is proposed. This method is illustrated with an example in a cancer clinical trial. A FORTRAN program for the computer implementation of the method is also discussed.     

A causal inference approach to measure price elasticity in Automobile Insurance

Understanding the precise nature of price sensitivities at the individual policyholder level is extremely valuable for insurers. A rate increase has a direct impact on the premium customers are paying, but there is also the indirect impact as a result of the “causal” effect of the rate change on the customer’s decision to renew the policy term. A rate increase may impair its intended impact on the overall profitability of the portfolio if it causes a large number of policyholders to lapse their policy and switch to an alternative insurer. The difficulty in measuring price elasticity from most insurance databases is that historical rate changes are reflective of a risk-based pricing exercise. As a result, the specific rate change at which a customer is exposed is a deterministic function of her observed covariates. The nature of the data is thus observational, rather than experimental. In this context, measuring the causal effect of a rate change on the policyholder’s lapse outcome requires special modeling considerations. Conventional modeling approaches aimed to directly fit the lapse outcome as a function of the rate change and background covariates are likely to be inappropriate for the problem at hand. In this paper, we propose a causal inference framework to measure price elasticity in the context of Auto Insurance. One of the strengths of our approach is the transparency about the extent to which the database can support causal effects from rate changes. The model also allows us to more reliably estimate price-elasticity functions at the individual policyholder level. As the causal effect of a rate change varies across individuals, making an accurate rate change choice at the individual subject level is essential. The rate at which each subject is exposed could be optimized on the basis of the individual characteristics, for the purpose of maximizing the overall expected profitability of the portfolio.     

Fuzzy causal networks: general model, inference, and convergence

In this paper, we first propose a general framework for fuzzy causal networks (FCNs). Then, we study the dynamics and convergence of such general FCNs. We prove that any general FCN with constant weight matrix converges to a limit cycle or a static state, or the trajectory of the FCN is not repetitive. We also prove that under certain conditions a discrete state general FCN converges to its limit cycle or static state in O(n) steps, where n is the number of vertices of the FCN. This is in striking contrast with the exponential running time 2/sup n/, which is accepted widely for classic FCNs.     

Learning and approximate inference in dynamic hierarchical models

A new variant of the dynamic hierarchical model (DHM) that describes a large number of parallel time series is presented. The separate series, which may be interdependent, are modeled through dynamic linear models (DLMs). This interdependence is included in the model through the definition of a ‘top-level’ or ‘average’ DLM. The model features explicit dependences between the latent states of the parallel DLMs and the states of the average model, and thus the many parallel time series are linked to each other. The combination of dependences within each time series and dependences between the different DLMs makes the computation time that is required for exact inference cubic in the number of parallel time series, however, which is unacceptable for practical tasks that involve large numbers of parallel time series. Therefore, two methods for fast, approximate inference are proposed: a variational approximation and a factorial approach. Under these approximations, inference can be performed in linear time, and it still features exact means. Learning is implemented through a maximum likelihood (ML) estimation of the model parameters. This estimation is realized through an expectation maximization (EM) algorithm with approximate inference in the E-step. Examples of learning and forecasting on two data sets show that the addition of direct dependences has a ‘smoothing’ effect on the evolution of the states of the individual time series, and leads to better prediction results. The use of approximate instead of exact inference is further shown not to lead to inferior results on either data set.     

A novel subspace identification approach with enforced causal models

Subspace identification methods (SIMs) for estimating state-space models have been proven to be very useful and numerically efficient. They exist in several variants, but all have one feature in common: as a first step, a collection of high-order ARX models are estimated from vectorized input–output data. In order not to obtain biased estimates, this step must include future outputs. However, all but one of the submodels include non-causal input terms. The coefficients of them will be correctly estimated to zero as more data become available. They still include extra model parameters which give unnecessarily high variance, and also cause bias for closed-loop data. In this paper, a new model formulation is suggested that circumvents the problem. Within the framework, the system matrices (A,B,C,D) and Markov parameters can be estimated separately. It is demonstrated through analysis that the new methods generally give smaller variance in the estimate of the observability matrix and it is supported by simulation studies that this gives lower variance also of the system invariants such as the poles.     

Bayesian estimation of beta mixture models with variational inference

Bayesian estimation of the parameters in beta mixture models (BMM) is analytically intractable. The numerical solutions to simulate the posterior distribution are available, but incur high computational cost. In this paper, we introduce an approximation to the prior/posterior distribution of the parameters in the beta distribution and propose an analytically tractable (closed form) Bayesian approach to the parameter estimation. The approach is based on the variational inference (VI) framework. Following the principles of the VI framework and utilizing the relative convexity bound, the extended factorized approximation method is applied to approximate the distribution of the parameters in BMM. In a fully Bayesian model where all of the parameters of the BMM are considered as variables and assigned proper distributions, our approach can asymptotically find the optimal estimate of the parameters posterior distribution. Also, the model complexity can be determined based on the data. The closed-form solution is proposed so that no iterative numerical calculation is required. Meanwhile, our approach avoids the drawback of overfitting in the conventional expectation maximization algorithm. The good performance of this approach is verified by experiments with both synthetic and real data.