Approximately calibrated small sample inference about means from bivariate normal data with missing values

Frequency properties of approximate Bayesian posterior probability intervals are considered for a small bivariate sample with data missing on one variable. For a class of priors, approximations to the posterior distribution based on matching moments of the t distribution are derived, and compared with the true distributions computed numerically. Coverage properties of highest posterior density intervals for three choices of prior are evaluated by simulation, and compared with other solutions. The simulations suggest that a second moment t approximation combined with the Jeffreys' prior for the bivariate distribution provides intervals that are quite well calibrated, in the sense of having approximate or slightly conservative coverage for a wide range of values of the underlying parameters. The use of calibration to select a suitable reference prior seems to have potential for a large number of problems.     

Dirichlet Process Gaussian Mixture Models: Choice of the Base Distribution

In the Bayesian mixture modeling framework it is possible to infer the necessary number of components to model the data and therefore it is unnecessary to explicitly restrict the number of components. Nonparametric mixture models sidestep the problem of finding the “correct” number of mixture components by assuming infinitely many components. In this paper Dirichlet process mixture (DPM) models are cast as infinite mixture models and inference using Markov chain Monte Carlo is described. The specification of the priors on the model parameters is often guided by mathematical and practical convenience. The primary goal of this paper is to compare the choice of conjugate and non-conjugate base distributions on a particular class of DPM models which is widely used in applications, the Dirichlet process Gaussian mixture model (DPGMM). We compare computational efficiency and modeling performance of DPGMM defined using a conjugate and a conditionally conjugate base distribution. We show that better density models can result from using a wider class of priors with no or only a modest increase in computational effort.     

Heterogeneous data analysis using a mixture of Laplace models with conjugate priors

The development of flexible parametric classes of probability models in Bayesian analysis is a very popular approach. This study is designed for heterogeneous population for a two-component mixture of the Laplace probability distribution. When a process initially starts, the researcher expects that the failure components will be very high but after some improvement/inspection it is assumed that the failure components will decrease sufficiently. That is why in such situation the Laplace model is more suitable as compared to the normal distribution due to its fatter tails behaviour. We considered the derivation of the posterior distribution for censored data assuming different conjugate informative priors. Various kinds of loss functions are used to derive these Bayes estimators and their posterior risks. A method of elicitation of hyperparameter is discussed based on a prior predictive approach. The results are also compared with the non-informative priors. To examine the performance of these estimators we have evaluated their properties for different sample sizes, censoring rates and proportions of the component of the mixture through the simulation study. To highlight the practical significance we have included an illustrative application example based on real-life mixture data.     

Estimation of generalized multisensor hidden Markov chains andunsupervised image segmentation

This paper attacks the problem of generalized multisensor mixture estimation. A distribution mixture is said to be generalized when the exact nature of components is not known, but each of them belongs to a finite known set of families of distributions. Estimating such a mixture entails a supplementary difficulty: one must label, for each class and each sensor, the exact nature of the corresponding distribution. Such generalized mixtures have been studied assuming that the components lie in the Pearson system. We propose a more general procedure with applications to estimating generalized multisensor hidden Markov chains. Our proposed method is applied to the problem of unsupervised image segmentation. The method proposed allows one to: 1) identify the conditional distribution for each class and each sensor, 2) estimate the unknown parameters in this distribution, 3) estimate priors, and 4) estimate the “true” class image     

Generalized exponential distribution: Bayesian estimations

Recently two-parameter generalized exponential distribution has been introduced by the authors. In this paper we consider the Bayes estimators of the unknown parameters under the assumptions of gamma priors on both the shape and scale parameters. The Bayes estimators cannot be obtained in explicit forms. Approximate Bayes estimators are computed using the idea of Lindley. We also propose Gibbs sampling procedure to generate samples from the posterior distributions and in turn computing the Bayes estimators. The approximate Bayes estimators obtained under the assumptions of non-informative priors, are compared with the maximum likelihood estimators using Monte Carlo simulations. One real data set has been analyzed for illustrative purposes.     

Bayesian combination of sparse and non-sparse priors in image super resolution

In this paper the application of image prior combinations to the Bayesian Super Resolution (SR) image registration and reconstruction problem is studied. Two sparse image priors, a Total Variation (TV) prior and a prior based on the ?1 norm of horizontal and vertical first-order differences (f.o.d.), are combined with a non-sparse Simultaneous Auto Regressive (SAR) prior. Since, for a given observation model, each prior produces a different posterior distribution of the underlying High Resolution (HR) image, the use of variational approximation will produce as many posterior approximations as priors we want to combine. A unique approximation is obtained here by finding the distribution on the HR image given the observations that minimizes a linear convex combination of Kullback–Leibler (KL) divergences. We find this distribution in closed form. The estimated HR images are compared with the ones obtained by other SR reconstruction methods.     

Transfer estimation of evolving class priors in data stream classification

Data stream classification is a hot topic in data mining research. The great challenge is that the class priors may evolve along the data sequence. Algorithms have been proposed to estimate the dynamic class priors and adjust the classifier accordingly. However, the existing algorithms do not perform well on prior estimation due to the lack of samples from the target distribution. Sample size has great effects in parameter estimation and small-sample effects greatly contaminate the estimation performance. In this paper, we propose a novel parameter estimation method called transfer estimation. Transfer estimation makes use of samples not only from the target distribution but also from similar distributions. We apply this new estimation method to the existing algorithms and obtain an improved algorithm. Experiments on both synthetic and real data sets show that the improved algorithm outperforms the existing algorithms on both class prior estimation and classification.     

A new generalization of the Waring distribution

A tetraparametric univariate distribution generated by the Gaussian hypergeometric function that includes the Waring and the generalized Waring distributions as particular cases is presented. This distribution is expressed as a generalized beta type I mixture of a negative binomial distribution, in such a way that the variance of the tetraparametric model can be split into three components: randomness, proneness and liability. These results are extensions of known analogous properties of the generalized Waring distribution. Two applications in the fields of sport and economy are included in order to illustrate the utility of the new distribution compared with the generalized Waring distribution.     

Step-up and step-down procedures controlling the number and proportion of false positives

In multiple hypotheses testing, it is important to control the probability of rejecting “true” null hypotheses. A standard procedure has been to control the family-wise error rate (FWER), the probability of rejecting at least one true null hypothesis.For large numbers of hypotheses, using FWER can result in very low power for testing single hypotheses. Recently, powerful multiple step FDR procedures have been proposed which control the “false discovery rate” (expected proportion of Type I errors). More recently, van der Laan et al. [Augmentation procedures for control of the generalized family-wise error rate and tail probabilities for the proportion of false positives. Statist. Appl. in Genetics and Molecular Biol. 3, 1-25] proposed controlling a generalized family-wise error rate k-FWER (also called gFWER(k)), defined as the probability of at least (k+1) Type I errors (k=0 for the usual FWER).Lehmann and Romano [Generalizations of the familywise error rate. Ann. Statist. 33(3), 1138-1154] suggested both a single-step and a step-down procedure for controlling the generalized FWER. They make no assumptions concerning the p-values of the individual tests. The step-down procedure is simple to apply, and cannot be improved without violation of control of the k-FWER.In this paper, by limiting the number of steps in step-down or step-up procedures, new procedures are developed to control k-FWER (and the proportion of false positives) PFP. Using data from the literature, the procedures are compared with those of Lehmann and Romano [Generalizations of the familywise error rate. Ann. Statist. 33(3), 1138-1154], and, under the assumption of a multivariate normal distribution of the test statistics, show considerable improvement in the reduction of the number and PFP.     

Flexible modelling of random effects in linear mixed models—A Bayesian approach

Flexible modelling of random effects in linear mixed models has attracted some attention recently. In this paper, we propose the use of finite Gaussian mixtures as in Verbeke and Lesaffre [A linear mixed model with heterogeneity in the random-effects population, J. Amu. Statist. Assoc. 91, 217-221]. We adopt a fully Bayesian hierarchical framework that allows simultaneous estimation of the number of mixture components together with other model parameters. The technique employed is the Reversible Jump MCMC algorithm (Richardson and Green [On Bayesian Analysis of Mixtures with an Unknown Number of Components (with discussion). J. Roy. Statist. Soc. Ser. B 59, 731-792]). This approach has the advantage of producing a direct comparison of different mixture models through posterior probabilities from a single run of the MCMC algorithm. Moreover, the Bayesian setting allows us to integrate over different mixture models to obtain a more robust density estimate of the random effects. We focus on linear mixed models with a random intercept and a random slope. Numerical results on simulated data sets and a real data set are provided to demonstrate the usefulness of the proposed method.     

Efficient Markov chain Monte Carlo methods for decoding neural spike trains

Stimulus reconstruction or decoding methods provide an important tool for understanding how sensory and motor information is represented in neural activity. We discuss Bayesian decoding methods based on an encoding generalized linear model (GLM) that accurately describes how stimuli are transformed into the spike trains of a group of neurons. The form of the GLM likelihood ensures that the posterior distribution over the stimuli that caused an observed set of spike trains is log concave so long as the prior is. This allows the maximum a posteriori (MAP) stimulus estimate to be obtained using efficient optimization algorithms. Unfortunately, the MAP estimate can have a relatively large average error when the posterior is highly nongaussian. Here we compare several Markov chain Monte Carlo (MCMC) algorithms that allow for the calculation of general Bayesian estimators involving posterior expectations (conditional on model parameters). An efficient version of the hybrid Monte Carlo (HMC) algorithm was significantly superior to other MCMC methods for gaussian priors. When the prior distribution has sharp edges and corners, on the other hand, the "hit-and-run" algorithm performed better than other MCMC methods. Using these algorithms, we show that for this latter class of priors, the posterior mean estimate can have a considerably lower average error than MAP, whereas for gaussian priors, the two estimators have roughly equal efficiency. We also address the application of MCMC methods for extracting nonmarginal properties of the posterior distribution. For example, by using MCMC to calculate the mutual information between the stimulus and response, we verify the validity of a computationally efficient Laplace approximation to this quantity for gaussian priors in a wide range of model parameters; this makes direct model-based computation of the mutual information tractable even in the case of large observed neural populations, where methods based on binning the spike train fail. Finally, we consider the effect of uncertainty in the GLM parameters on the posterior estimators.     

A geometric view of conjugate priors

In Bayesian machine learning, conjugate priors are popular, mostly due to mathematical convenience. In this paper, we show that there are deeper reasons for choosing a conjugate prior. Specifically, we formulate the conjugate prior in the form of Bregman divergence and show that it is the inherent geometry of conjugate priors that makes them appropriate and intuitive. This geometric interpretation allows one to view the hyperparameters of conjugate priors as the effective sample points, thus providing additional intuition. We use this geometric understanding of conjugate priors to derive the hyperparameters and expression of the prior used to couple the generative and discriminative components of a hybrid model for semi-supervised learning.     

A consistent nonparametric Bayesian procedure for estimating autoregressive conditional densities

This article proposes a Bayesian infinite mixture model for the estimation of the conditional density of an ergodic time series. A nonparametric prior on the conditional density is described through the Dirichlet process. In the mixture model, a kernel is used leading to a dynamic nonlinear autoregressive model. This model can approximate any linear autoregressive model arbitrarily closely while imposing no constraint on parameters to ensure stationarity. We establish sufficient conditions for posterior consistency in two different topologies. The proposed method is compared with the mixture of autoregressive model [Wong and Li, 2000. On a mixture autoregressive model. J. Roy. Statist. Soc. Ser. B 62(1), 91-115] and the double-kernel local linear approach [Fan et al., 1996. Estimation of conditional densities and sensitivity measures in nonlinear dynamical systems. Biometrika 83, 189-206] by simulations and real examples. Our method shows excellent performances in these studies.     

Surface reconstruction with data-driven exemplar priors

In this paper, we propose a framework to reconstruct 3D models from raw scanned points by learning the prior knowledge of a specific class of objects. Unlike previous work that heuristically specifies particular regularities and defines parametric models, our shape priors are learned directly from existing 3D models under a framework based on affinity propagation. Given a database of 3D models within the same class of objects, we build a comprehensive library of 3D local shape priors. We then formulate the problem to select as-few-as-possible priors from the library, referred to as exemplar priors. These priors are sufficient to represent the 3D shapes of the whole class of objects from where they are generated. By manipulating these priors, we are able to reconstruct geometrically faithful models with the same class of objects from raw point clouds. Our framework can be easily generalized to reconstruct various categories of 3D objects that have more geometrically or topologically complex structures. Comprehensive experiments exhibit the power of our exemplar priors for gracefully solving several problems in 3D shape reconstruction such as preserving sharp features, recovering fine details and so on.     

Bayesian analysis of Birnbaum-Saunders distribution with partial information

In Bayesian analysis with objective priors, it should be justified that the posterior distribution is proper. In this paper, we show that the reference prior (or independent Jeffreys prior) of a two-parameter Birnbaum-Saunders distribution will result in an improper posterior distribution. However, the posterior distributions are proper based on the reference priors with partial information (RPPI). Based on censored samples, slice sampling is utilized to obtain the Bayesian estimators based on RPPI. Monte Carlo simulations are used to compare the efficiencies of different RPPIs, to assess the sensitivity of the choice of the priors, and to compare the Bayesian estimators with the maximum likelihood estimators, for various scales of sample size and degree of censoring. A real data set is analyzed for illustrative purpose.     

Using the EM algorithm for inference in a mixture of distributions with censored but partially identifiable data

This paper describes an implementation of the EM algorithm for the statistical analysis of a finite mixture of distributions arising when data are censored but partially identifiable. We consider a scheme of type I censoring where censoring times are random. The estimation of standard errors proposed by Meng and Rubin (1991. Using EM to obtain asymptotic variance-covariance matrices: the SEM algorithm. J. Amer. Statist. Assoc. 86(416), 899-909) is also implemented in the context of the above mixture. A Bayesian method introduced in Contreras-Cristán et al. (2003. Statistical inference for mixtures of distributions for censored data with partial identification. Commun. in Statist. Theory Methods 32(4), 749-774) for the case of a constant censoring value is extended to the case of random censoring times. Comparisons with different methods are carried out both with simulated data and with the observations on failure times for communication transmitter-receivers of Mendenhall and Hader (1958. Estimation of parameters of mixed exponentially distributed failure time distributions from censored life test data. Biometrika 45, 504-520).     

On Bayesian analysis of a finite generalized Dirichlet mixture via a Metropolis-within-Gibbs sampling

In this paper, we present a fully Bayesian approach for generalized Dirichlet mixtures estimation and selection. The estimation of the parameters is based on the Monte Carlo simulation technique of Gibbs sampling mixed with a Metropolis-Hastings step. Also, we obtain a posterior distribution which is conjugate to a generalized Dirichlet likelihood. For the selection of the number of clusters, we used the integrated likelihood. The performance of our Bayesian algorithm is tested and compared with the maximum likelihood approach by the classification of several synthetic and real data sets. The generalized Dirichlet mixture is also applied to the problems of IR eye modeling and introduced as a probabilistic kernel for Support Vector Machines.

Interval estimation of binomial proportions based on weighted Polya posterior

Recently the interval estimation of binomial proportions is revisited in various literatures. This is mainly due to the erratic behavior of the coverage probability of the well-known Wald confidence interval. Various alternatives have been proposed. Among them, Agresti-Coull confidence interval has been recommended by Brown et al. [2001. Interval estimation for a binomial proportion. Statist. Sci. 16, 101-133] with other confidence intervals such as the Wilson interval and the equal tailed interval resulting from the natural noninformative Jefferys prior for a binomial proportion. However, it seems that Agresti-Coull interval is little bit wider than necessary when sample size is small, say n?40. In this note, an interval estimator is developed using weighted Polya posterior. It is shown that the confidence interval based on the weighted Polya posterior is essentially the Agresti-Coull interval with some improved features.     

A framework for modelling overdispersed count data, including the Poisson-shifted generalized inverse Gaussian distribution

A variety of methods of modelling overdispersed count data are compared. The methods are classified into three main categories. The first category are ad hoc methods (i.e. pseudo-likelihood, (extended) quasi-likelihood, double exponential family distributions). The second category are discretized continuous distributions and the third category are observational level random effects models (i.e. mixture models comprising explicit and non-explicit continuous mixture models and finite mixture models). The main focus of the paper is a family of mixed Poisson distributions defined so that its mean μ is an explicit parameter of the distribution. This allows easier interpretation when μ is modelled using explanatory variables and provides a more orthogonal parameterization to ease model fitting. Specific three parameter distributions considered are the Sichel and Delaporte distributions. A new four parameter distribution, the Poisson-shifted generalized inverse Gaussian distribution is introduced, which includes the Sichel and Delaporte distributions as a special and a limiting case respectively. A general formula for the derivative of the likelihood with respect to μ, applicable to the whole family of mixed Poisson distributions considered, is given. Within the framework introduced here all parameters of the distributions are modelled as parametric and/or nonparametric (smooth) functions of explanatory variables. This provides a very flexible way of modelling count data. Maximum (penalized) likelihood estimation is used to fit the (non)parametric models.     

Minimax classifiers based on neural networks

The problem of designing a classifier when prior probabilities are not known or are not representative of the underlying data distribution is discussed in this paper. Traditional learning approaches based on the assumption that class priors are stationary lead to sub-optimal solutions if there is a mismatch between training and future (real) priors. To protect against this uncertainty, a minimax approach may be desirable. We address the problem of designing a neural-based minimax classifier and propose two different algorithms: a learning rate scaling algorithm and a gradient-based algorithm. Experimental results show that both succeed in finding the minimax solution and it is also pointed out the differences between common approaches to cope with this uncertainty in priors and the minimax classifier.