Online Bayesian inference for the parameters of PRISM programs

This paper presents a method for approximating posterior distributions over the parameters of a given PRISM program. A sequential approach is taken where the distribution is updated one datapoint at a time. This makes it applicable to online learning situations where data arrives over time. The method is applicable whenever the prior is a mixture of products of Dirichlet distributions. In this case the true posterior will be a mixture of very many such products. An approximation is effected by merging products of Dirichlet distributions. An analysis of the quality of the approximation is presented. Due to the heavy computational burden of this approach, the method has been implemented in the Mercury logic programming language. Initial results using a hidden Markov model and a probabilistic graph are presented.     

A unified Bayesian inference on treatment means with order constraints

In some applications involving comparison of treatment means, it is known a priori that population means are ordered in a certain way. In such situations, imposing constraints on the treatment means can greatly increase the effectiveness of statistical procedures.This paper proposes a unified Bayesian method which performs a simultaneous comparison of treatment means and parameter estimation in ANOVA models with order constraints on the means. A continuous prior restricted to order constraints is employed, and posterior samples of parameters are generated using a Markov chain Monte Carlo method. Posterior probabilities of all possible hypotheses on the equality/inequality of treatment means are obtained using Savage-Dickey density ratios, for which we propose a simple and computationally efficient estimation method. Posterior densities and HPD intervals of parameters of interest are estimated with almost no extra cost, given some by-products from the test procedure.Simulation study results show that the proposed method outperforms the test without constraints and that the method is powerful in detecting the true hypothesis. The method is applied to the ramus bone sizes of 20 boys, which were measured at four time points. The proposed Bayesian test reveals that there are two growth spurts in the ramus bone size during the observed period, which could not be detected by pairwise comparisons of the means.     

Bayesian inference for differential equations

Nonlinear dynamic systems such as biochemical pathways can be represented in abstract form using a number of modelling formalisms. In particular differential equations provide a highly expressive mathematical framework with which to model dynamic systems, and a very natural way to model the dynamics of a biochemical pathway in a deterministic manner is through the use of nonlinear ordinary or time delay differential equations. However if, for example, we consider a biochemical pathway the constituent chemical species and hence the pathway structure are seldom fully characterised. In addition it is often impossible to obtain values of the rates of activation or decay which form the free parameters of the mathematical model. The system model in many cases is therefore not fully characterised either in terms of structure or the values which parameters take. This uncertainty must be accounted for in a systematic manner when the model is used in simulation or predictive mode to safeguard against reaching conclusions about system characteristics that are unwarranted, or in making predictions that are unjustifiably optimistic given the uncertainty about the model. The Bayesian inferential methodology provides a coherent framework with which to characterise and propagate uncertainty in such mechanistic models and this paper provides an introduction to Bayesian methodology as applied to system models represented as differential equations.     

Bayesian inference for wind field retrieval

In many problems in spatial statistics it is necessary to infer a global problem solution by combining local models. A principled approach to this problem is to develop a global probabilistic model for the relationships between local variables and to use this as the prior in a Bayesian inference procedure. We use a Gaussian process with hyper-parameters estimated from numerical weather prediction models, which yields meteorologically convincing wind fields. We use neural networks to make local estimates of wind vector probabilities. The resulting inference problem cannot be solved analytically, but Markov Chain Monte Carlo methods allow us to retrieve accurate wind fields.     

Bayesian spiking neurons I: inference

We show that the dynamics of spiking neurons can be interpreted as a form of Bayesian inference in time. Neurons that optimally integrate evidence about events in the external world exhibit properties similar to leaky integrate-and-fire neurons with spike-dependent adaptation and maximally respond to fluctuations of their input. Spikes signal the occurrence of new information-what cannot be predicted from the past activity. As a result, firing statistics are close to Poisson, albeit providing a deterministic representation of probabilities.     

Structure inference for Bayesian multisensory scene understanding

We investigate a solution to the problem of multi-sensor scene understanding by formulating it in the framework of Bayesian model selection and structure inference. Humans robustly associate multimodal data as appropriate, but previous modelling work has focused largely on optimal fusion, leaving segregation unaccounted for and unexploited by machine perception systems. We illustrate a unifying, Bayesian solution to multi-sensor perception and tracking which accounts for both integration and segregation by explicit probabilistic reasoning about data association in a temporal context. Such explicit inference of multimodal data association is also of intrinsic interest for higher level understanding of multisensory data. We illustrate this using a probabilistic implementation of data association in a multi-party audio-visual scenario, where unsupervised learning and structure inference is used to automatically segment, associate and track individual subjects in audiovisual sequences. Indeed, the structure inference based framework introduced in this work provides the theoretical foundation needed to satisfactorily explain many confounding results in human psychophysics experiments involving multimodal cue integration and association.     

Approximate inference for dynamic Bayesian networks: sliding window approach

Dynamic Bayesian networks (DBNs) are probabilistic graphical models that have become a ubiquitous tool for compactly describing statistical relationships among a group of stochastic processes. A suite of elaborately designed inference algorithms makes it possible for intelligent systems to use a DBN to make inferences in uncertain conditions. Unfortunately, exact inference or even approximation in a DBN has been proved to be NP-hard and is generally computationally prohibitive. In this paper, we investigate a sliding window framework for approximate inference in DBNs to reduce the computational burden. By introducing a sliding window that moves forward as time progresses, inference at any time is restricted to a quite narrow region of the network. The main contributions to the sliding window framework include an exploration of its foundations, explication of how it operates, and the proposal of two strategies for adaptive window size selection. To make this framework available as an inference engine, the interface algorithm widely used in exact inference is then integrated with the framework for approximate inference in DBNs. After analyzing its computational complexity, further empirical work is presented to demonstrate the validity of the proposed algorithms.     

Approximate posterior inference for Bayesian models: black-box expectation propagation

Knowledge and Information Systems - Expectation propagation (EP) is a widely successful way to approximate the posteriors of complex Bayesian models. However, it suffers from expensive memory and...     

Bayesian inference for additive mixed quantile regression models

Quantile regression problems in practice may require flexible semiparametric forms of the predictor for modeling the dependence of responses on covariates. Furthermore, it is often necessary to add random effects accounting for overdispersion caused by unobserved heterogeneity or for correlation in longitudinal data. We present a unified approach for Bayesian quantile inference on continuous response via Markov chain Monte Carlo (MCMC) simulation and approximate inference using integrated nested Laplace approximations (INLA) in additive mixed models. Different types of covariate are all treated within the same general framework by assigning appropriate Gaussian Markov random field (GMRF) priors with different forms and degrees of smoothness. We applied the approach to extensive simulation studies and a Munich rental dataset, showing that the methods are also computationally efficient in problems with many covariates and large datasets.     

Sparse Bayesian learning for efficient visual tracking

This paper extends the use of statistical learning algorithms for object localization. It has been shown that object recognizers using kernel-SVMs can be elegantly adapted to localization by means of spatial perturbation of the SVM. While this SVM applies to each frame of a video independently of other frames, the benefits of temporal fusion of data are well-known. This is addressed here by using a fully probabilistic relevance vector machine (RVM) to generate observations with Gaussian distributions that can be fused over time. Rather than adapting a recognizer, we build a displacement expert which directly estimates displacement from the target region. An object detector is used in tandem, for object verification, providing the capability for automatic initialization and recovery. This approach is demonstrated in real-time tracking systems where the sparsity of the RVM means that only a fraction of CPU time is required to track at frame rate. An experimental evaluation compares this approach to the state of the art showing it to be a viable method for long-term region tracking.     

Approximate Bayesian methods for kernel-based object tracking

A framework for real-time tracking of complex non-rigid objects is presented. The object shape is approximated by an ellipse and its appearance by histogram based features derived from local image properties. An efficient search procedure is used to find the image region with a histogram most similar to the histogram of the tracked object. The procedure is a natural extension of the mean-shift procedure with Gaussian kernel which allows handling the scale and orientation changes of the object. The presented procedure is integrated into a set of Bayesian filtering schemes. We compare the regular and mixture Kalman filter and other sequential importance sampling (particle filtering) techniques.     

Bayesian A* tree search with expected O(N) node expansions: applications to road tracking

Many perception, reasoning, and learning problems can be expressed as Bayesian inference. We point out that formulating a problem as Bayesian inference implies specifying a probability distribution on the ensemble of problem instances. This ensemble can be used for analyzing the expected complexity of algorithms and also the algorithm-independent limits of inference. We illustrate this problem by analyzing the complexity of tree search. In particular, we study the problem of road detection, as formulated by Geman and Jedynak (1996). We prove that the expected convergence is linear in the size of the road (the depth of the tree) even though the worst-case performance is exponential. We also put a bound on the constant of the convergence and place a bound on the error rates.     

Mean-field variational approximate Bayesian inference for latent variable models

The ill-posed nature of missing variable models offers a challenging testing ground for new computational techniques. This is the case for the mean-field variational Bayesian inference. The behavior of this approach in the setting of the Bayesian probit model is illustrated. It is shown that the mean-field variational method always underestimates the posterior variance and, that, for small sample sizes, the mean-field variational approximation to the posterior location could be poor.     

Mobile context inference using two-layered Bayesian networks for smartphones

Recently, mobile context inference becomes an important issue. Bayesian probabilistic model is one of the most popular probabilistic approaches for context inference. It efficiently represents and exploits the conditional independence of propositions. However, there are some limitations for probabilistic context inference in mobile devices. Mobile devices relatively lacks of sufficient memory. In this paper, we present a novel method for efficient Bayesian inference on a mobile phone. In order to overcome the constraints of the mobile environment, the method uses two-layered Bayesian networks with tree structure. In contrast to the conventional techniques, this method attempts to use probabilistic models with fixed tree structures and intermediate nodes. It can reduce the inference time by eliminating junction tree creation. To evaluate the performance of this method, an experiment is conducted with data collected over a month. The result shows the efficiency and effectiveness of the proposed method.     

Pseudo-marginal Bayesian inference for Gaussian process latent variable models

Machine Learning - A Bayesian inference framework for supervised Gaussian process latent variable models is introduced. The framework overcomes the high correlations between latent variables and...     

Active tracking strategy for monocular depth inference overmultiple frames

The extraction of depth information from a sequence of images is investigated. An algorithm that exploits the constraint imposed by active motion of the camera is described. Within this framework, in order to facilitate measurement of the navigation parameters, a constrained egomotion strategy was adopted in which the position of the fixation point is stabilized during the navigation (in an anthropomorphic fashion). This constraint reduces the dimensionality of the parameter space without increasing the complexity of the equations. A further distinctive point is the use of two sampling rates: the faster (related to the computation of the instantaneous optical flow) is fast enough to allow the local operator to sense the passing edge (or, in other words, to allow the tracking of moving contour points), while the slower (used to perform the triangulation procedure necessary to derive depth) is slow enough to provide a sufficiently large baseline for triangulation. Experimental results on real image sequences are presented     

Multiple Condensation filters for road detection and tracking

In this paper, we present a novel and robust road tracking system for vision-based personal navigation. Novelty of the work includes the use of multiple Condensation filters to track the road of arbitrary shape and automatic switching between trackers according to road conditions. The approach allows the road to be represented as a simple hyperbola. It also supports the representation of the road as a sequence of connected arcs/segments so that information from a digital map can be integrated into tracking. The parameters of the hyperbola road model are estimated using multiple vanishing points located in image strips. The road tracking method is robust in dealing with complex road shapes, background clutters, shadows, and road markings. Experiments using real videos demonstrate the robustness of our approach.     

Bayesian inference for an item response model for modeling test anxiety

We develop a Bayesian binary Item Response Model (IRM), which we denote as Test Anxiety Model (TAM), for estimating the proficiency scores when individuals might experience test anxiety. We consider order restricted item parameters conditionally to the examinees’ reported emotional state at the testing session. We consider three test anxiety levels: calm, anxious and very anxious. Using simulated data we show that taking into account test anxiety levels in an IRM help us to obtain fair proficiency estimates as opposed to the ones obtained with three two-parameter logistic IRM (3PM) by Birnbaum (1957, 1968). For the 3PM, the proficiency estimates tend to be positively biased for both, calm and anxious examinees.     

Manhattan world: orientation and outlier detection by Bayesian inference

This letter argues that many visual scenes are based on a "Manhattan" three-dimensional grid that imposes regularities on the image statistics. We construct a Bayesian model that implements this assumption and estimates the viewer orientation relative to the Manhattan grid. For many images, these estimates are good approximations to the viewer orientation (as estimated manually by the authors). These estimates also make it easy to detect outlier structures that are unaligned to the grid. To determine the applicability of the Manhattan world model, we implement a null hypothesis model that assumes that the image statistics are independent of any three-dimensional scene structure. We then use the log-likelihood ratio test to determine whether an image satisfies the Manhattan world assumption. Our results show that if an image is estimated to be Manhattan, then the Bayesian model's estimates of viewer direction are almost always accurate (according to our manual estimates), and vice versa.