Model-based decoding, information estimation, and change-point detection techniques for multineuron spike trains

One of the central problems in systems neuroscience is to understand how neural spike trains convey sensory information. Decoding methods, which provide an explicit means for reading out the information contained in neural spike responses, offer a powerful set of tools for studying the neural coding problem. Here we develop several decoding methods based on point-process neural encoding models, or forward models that predict spike responses to stimuli. These models have concave log-likelihood functions, which allow efficient maximum-likelihood model fitting and stimulus decoding. We present several applications of the encoding model framework to the problem of decoding stimulus information from population spike responses: (1) a tractable algorithm for computing the maximum a posteriori (MAP) estimate of the stimulus, the most probable stimulus to have generated an observed single- or multiple-neuron spike train response, given some prior distribution over the stimulus; (2) a gaussian approximation to the posterior stimulus distribution that can be used to quantify the fidelity with which various stimulus features are encoded; (3) an efficient method for estimating the mutual information between the stimulus and the spike trains emitted by a neural population; and (4) a framework for the detection of change-point times (the time at which the stimulus undergoes a change in mean or variance) by marginalizing over the posterior stimulus distribution. We provide several examples illustrating the performance of these estimators with simulated and real neural data.     

Computational tools for comparing asymmetric GARCH models via Bayes factors

In this paper we use Markov chain Monte Carlo (MCMC) methods in order to estimate and compare GARCH models from a Bayesian perspective. We allow for possibly heavy tailed and asymmetric distributions in the error term. We use a general method proposed in the literature to introduce skewness into a continuous unimodal and symmetric distribution. For each model we compute an approximation to the marginal likelihood, based on the MCMC output. From these approximations we compute Bayes factors and posterior model probabilities.     

Bayesian analysis of Birnbaum-Saunders distribution with partial information

In Bayesian analysis with objective priors, it should be justified that the posterior distribution is proper. In this paper, we show that the reference prior (or independent Jeffreys prior) of a two-parameter Birnbaum-Saunders distribution will result in an improper posterior distribution. However, the posterior distributions are proper based on the reference priors with partial information (RPPI). Based on censored samples, slice sampling is utilized to obtain the Bayesian estimators based on RPPI. Monte Carlo simulations are used to compare the efficiencies of different RPPIs, to assess the sensitivity of the choice of the priors, and to compare the Bayesian estimators with the maximum likelihood estimators, for various scales of sample size and degree of censoring. A real data set is analyzed for illustrative purpose.     

Computational Issues in the Sequential Probit Model: A Monte Carlo Study

We discuss computational issues in the sequential probit model that have limited its use in applied research. We estimate parameters of the model by the method of simulated maximum likelihood (SML) and by Bayesian MCMC algorithms. We provide Monte Carlo evidence on the relative performance of both estimators and find that the SML procedure computes standard errors of the estimated correlation coefficients that are less reliable. Given the numerical difficulties associated with the estimation procedures, we advise the applied researcher to use both the stochastic optimization algorithm in the Simulated Maximum Likelihood approach and the Bayesian MCMC algorithm to check the compatibility of the results. JEL Classifications: C11, C15, C35, C63     

Fast sampling methods for Bayesian max-margin models

Bayesian max-margin models have shown superiority in various practical applications, such as text categorization, collaborative prediction, social network link prediction and crowdsourcing, and they conjoin the flexibility of Bayesian modeling and predictive strengths of max-margin learning. However, Monte Carlo sampling for these models still remains challenging, especially for applications that involve large-scale datasets. In this paper, we present the stochastic subgradient Hamiltonian Monte Carlo (HMC) methods, which are easy to implement and computationally efficient. We show the approximate detailed balance property of subgradient HMC which reveals a natural and validated generalization of the ordinary HMC. Furthermore, we investigate the variants that use stochastic subsampling and thermostats for better scalability and mixing. Using stochastic subgradient Markov Chain Monte Carlo (MCMC), we efficiently solve the posterior inference task of various Bayesian max-margin models and extensive experimental results demonstrate the effectiveness of our approach.     

Variational Bayesian methods for spatial data analysis

With scientific data available at geocoded locations, investigators are increasingly turning to spatial process models for carrying out statistical inference. However, fitting spatial models often involves expensive matrix decompositions, whose computational complexity increases in cubic order with the number of spatial locations. This situation is aggravated in Bayesian settings where such computations are required once at every iteration of the Markov chain Monte Carlo (MCMC) algorithms. In this paper, we describe the use of Variational Bayesian (VB) methods as an alternative to MCMC to approximate the posterior distributions of complex spatial models. Variational methods, which have been used extensively in Bayesian machine learning for several years, provide a lower bound on the marginal likelihood, which can be computed efficiently. We provide results for the variational updates in several models especially emphasizing their use in multivariate spatial analysis. We demonstrate estimation and model comparisons from VB methods by using simulated data as well as environmental data sets and compare them with inference from MCMC.     

Generalized exponential distribution: Bayesian estimations

Recently two-parameter generalized exponential distribution has been introduced by the authors. In this paper we consider the Bayes estimators of the unknown parameters under the assumptions of gamma priors on both the shape and scale parameters. The Bayes estimators cannot be obtained in explicit forms. Approximate Bayes estimators are computed using the idea of Lindley. We also propose Gibbs sampling procedure to generate samples from the posterior distributions and in turn computing the Bayes estimators. The approximate Bayes estimators obtained under the assumptions of non-informative priors, are compared with the maximum likelihood estimators using Monte Carlo simulations. One real data set has been analyzed for illustrative purposes.     

A Noninformative Prior for Neural Networks

While many implementations of Bayesian neural networks use large, complex hierarchical priors, in much of modern Bayesian statistics, noninformative (flat) priors are very common. This paper introduces a noninformative prior for feed-forward neural networks, describing several theoretical and practical advantages of this approach. In particular, a simpler prior allows for a simpler Markov chain Monte Carlo algorithm. Details of MCMC implementation are included.     

Front-view vehicle detection by Markov chain Monte Carlo method

In this paper, we propose a new vehicle detection approach based on Markov chain Monte Carlo (MCMC). We mainly discuss the detection of vehicles in front-view static images with frequent occlusions. Models of roads and vehicles based on edge information are presented, the Bayesian problem's formulations are constructed, and a Markov chain is designed to sample proposals to detect vehicles. Using the Monte Carlo technique, we detect vehicles sequentially based on the idea of maximizing a posterior probability (MAP), performing vehicle segmentation in the meantime. Our method does not require complex preprocessing steps such as background extraction or shadow elimination, which are required in many existing methods. Experimental results show that the method has a high detection rate on vehicles and can perform successful segmentation, and reduce the influence caused by vehicle occlusion.     

Default Bayesian model determination methods for generalised linear mixed models

A default strategy for fully Bayesian model determination for generalised linear mixed models (GLMMs) is considered which addresses the two key issues of default prior specification and computation. In particular, the concept of unit-information priors is extended to the parameters of a GLMM. A combination of Markov chain Monte Carlo (MCMC) and Laplace approximations is used to compute approximations to the posterior model probabilities to find a subset of models with high posterior model probability. Bridge sampling is then used on the models in this subset to approximate the posterior model probabilities more accurately. The strategy is applied to four examples.     

Classification of newborn EEG maturity with Bayesian averaging over decision trees

EEG experts can assess a newborn’s brain maturity by visual analysis of age-related patterns in sleep EEG. It is highly desirable to make the results of assessment most accurate and reliable. However, the expert analysis is limited in capability to provide the estimate of uncertainty in assessments. Bayesian inference has been shown providing the most accurate estimates of uncertainty by using Markov Chain Monte Carlo (MCMC) integration over the posterior distribution. The use of MCMC enables to approximate the desired distribution by sampling the areas of interests in which the density of distribution is high. In practice, the posterior distribution can be multimodal, and so that the existing MCMC techniques cannot provide the proportional sampling from the areas of interest. The lack of prior information makes MCMC integration more difficult when a model parameter space is large and cannot be explored in detail within a reasonable time. In particular, the lack of information about EEG feature importance can affect the results of Bayesian assessment of EEG maturity. In this paper we explore how the posterior information about EEG feature importance can be used to reduce a negative influence of disproportional sampling on the results of Bayesian assessment. We found that the MCMC integration tends to oversample the areas in which a model parameter space includes one or more features, the importance of which counted in terms of their posterior use is low. Using this finding, we proposed to cure the results of MCMC integration and then described the results of testing the proposed method on a set of sleep EEG recordings.     

Random effects in promotion time cure rate models

In this paper, a survival model with long-term survivors and random effects, based on the promotion time cure rate model formulation for models with a surviving fraction is investigated. We present Bayesian and classical estimation approaches. The Bayesian approach is implemented using a Markov chain Monte Carlo (MCMC) based on the Metropolis-Hastings algorithms. For the second one, we use restricted maximum likelihood (REML) estimators. A simulation study is performed to evaluate the accuracy of the applied techniques for the estimates and their standard deviations. An example on an oropharynx cancer study is used to illustrate the model and the estimation approaches considered in the study.     

Bayesian adaptive bandwidth kernel density estimation of irregular multivariate distributions

In this paper, we propose a new methodology for multivariate kernel density estimation in which data are categorized into low- and high-density regions as an underlying mechanism for assigning adaptive bandwidths. We derive the posterior density of the bandwidth parameters via the Kullback-Leibler divergence criterion and use a Markov chain Monte Carlo (MCMC) sampling algorithm to estimate the adaptive bandwidths. The resulting estimator is referred to as the tail-adaptive density estimator. Monte Carlo simulation results show that the tail-adaptive density estimator outperforms the global-bandwidth density estimators implemented using different global bandwidth selection rules. The inferential potential of the tail-adaptive density estimator is demonstrated by employing the estimator to estimate the bivariate density of daily index returns observed from the USA and Australian stock markets.     

Orthogonal parallel MCMC methods for sampling and optimization

Monte Carlo (MC) methods are widely used for Bayesian inference and optimization in statistics, signal processing and machine learning. A well-known class of MC methods are Markov Chain Monte Carlo (MCMC) algorithms. In order to foster better exploration of the state space, specially in high-dimensional applications, several schemes employing multiple parallel MCMC chains have been recently introduced. In this work, we describe a novel parallel interacting MCMC scheme, called orthogonal MCMC (O-MCMC), where a set of “vertical” parallel MCMC chains share information using some “horizontal” MCMC techniques working on the entire population of current states. More specifically, the vertical chains are led by random-walk proposals, whereas the horizontal MCMC techniques employ independent proposals, thus allowing an efficient combination of global exploration and local approximation. The interaction is contained in these horizontal iterations. Within the analysis of different implementations of O-MCMC, novel schemes in order to reduce the overall computational cost of parallel Multiple Try Metropolis (MTM) chains are also presented. Furthermore, a modified version of O-MCMC for optimization is provided by considering parallel Simulated Annealing (SA) algorithms. Numerical results show the advantages of the proposed sampling scheme in terms of efficiency in the estimation, as well as robustness in terms of independence with respect to initial values and the choice of the parameters.     

Oil spill detection from SAR intensity imagery using a marked point process

This paper presents a new algorithm for the detection of oil spill from SAR intensity images. The proposed algorithm combines the marked point process, Bayesian inference and Markov Chain Monte Carlo (MCMC) technique. In this paper, the candidates of oil spills or dark spots in a SAR intensity image are characterized by a Poisson marked point process. The marked point process is formed by a group of random points (as a point process modelling the locations of oil spills) and a set of parameters including geometric parameters of windows centred at the random points and gamma distribution parameters (as the marks attaching to each point). As a result, the candidates of oil spills are represented by a group of windows, in which the intensities of pixels follow independent and identical gamma distribution with lower mean than that for the identical gamma distribution of the pixels out of windows. Following the Bayesian paradigm, the posterior distribution, which characterizes the locations and statistical distributions of oil spills, can be obtained up to a normalizing constant. In order to simulate from the posterior distribution and to estimate the parameters of the posterior distribution, the Revisable Jump MCMC (RJMCMC) algorithm is used. The optimal locations and sizes of dark spots are obtained by a maximum a posteriori (MAP) algorithm. The proposed approach is tested using Radarsat-1 SAR images with oil spills indicated by human analysts. The results show that the proposed approach works well and is very promising.     

Variational EM method for blur estimation using the spike-and-slab image prior

Most state-of-the-art blind image deconvolution methods rely on the Bayesian paradigm to model the deblurring problem and estimate both the blur kernel and latent image. It is customary to model the image in the filter space, where it is supposed to be sparse, and utilize convenient priors to account for this sparsity. In this paper, we propose the use of the spike-and-slab prior together with an efficient variational Expectation Maximization (EM) inference scheme to estimate the blur in the image. The spike-and-slab prior, which constitutes the gold standard in sparse machine learning, selectively shrinks irrelevant variables while mildly regularizing the relevant ones. The proposed variational Expectation Maximization algorithm is more efficient than usual Markov Chain Monte Carlo (MCMC) inference and, also, proves to be more accurate than the standard mean-field variational approximation. Additionally, all the prior model parameters are estimated by the proposed scheme. After blur estimation, a non-blind restoration method is used to obtain the actual estimation of the sharp image. We investigate the behavior of the prior in the experimental section together with a series of experiments with synthetically generated and real blurred images that validate the method's performance in comparison with state-of-the-art blind deconvolution techniques.     

Bayesian analysis of autoregressive moving average processes with unknown orders

A Bayesian model selection for modelling a time series by an autoregressive-moving-average model (ARMA) is presented. The posterior distribution of unknown parameters and the selected orders are obtained by the Markov chain Monte Carlo (MCMC) method. An MCMC algorithm that represents the parameters of the model as a point process has been implemented. The method is illustrated on simulated series and a real dataset.     

Exchange Monte Carlo Sampling From Bayesian Posterior for Singular Learning Machines

Many singular learning machines such as neural networks, normal mixtures, Bayesian networks, and hidden Markov models belong to singular learning machines and are widely used in practical information systems. In these learning machines, it is well known that Bayesian learning provides better generalization performance than maximum-likelihood estimation. However, it needs huge computational cost to sample from a Bayesian posterior distribution of a singular learning machine by a conventional Markov chain Monte Carlo (MCMC) method, such as the metropolis algorithm, because of singularities. Recently, the exchange Monte Carlo (MC) method, which is well known as an improved algorithm of MCMC method, has been proposed to apply to Bayesian neural network learning in the literature. In this paper, we propose the idea that the exchange MC method has a better effect on Bayesian learning in singular learning machines than that in regular learning machines, and show its effectiveness by comparing the numerical stochastic complexity with the theoretical one.      

An approximate Bayesian approach for quantitative trait loci estimation

Bayesian approaches have been widely used in quantitative trait locus (QTL) linkage analysis in experimental crosses, and have advantages in interpretability and in constructing parameter probability intervals. Most existing Bayesian linkage methods involve Monte Carlo sampling, which is computationally prohibitive for high-throughput applications such as eQTL analysis. In this paper, we present a Bayesian linkage model that offers directly interpretable posterior densities or Bayes factors for linkage. For our model, we employ the Laplace approximation for integration over nuisance parameters in backcross (BC) and F2 intercross designs. Our approach is highly accurate, and very fast compared with alternatives, including grid search integration, importance sampling, and Markov Chain Monte Carlo (MCMC). Our approach is thus suitable for high-throughput applications. Simulated and real datasets are used to demonstrate our proposed approach.