Bayesian inference for differential equations

Nonlinear dynamic systems such as biochemical pathways can be represented in abstract form using a number of modelling formalisms. In particular differential equations provide a highly expressive mathematical framework with which to model dynamic systems, and a very natural way to model the dynamics of a biochemical pathway in a deterministic manner is through the use of nonlinear ordinary or time delay differential equations. However if, for example, we consider a biochemical pathway the constituent chemical species and hence the pathway structure are seldom fully characterised. In addition it is often impossible to obtain values of the rates of activation or decay which form the free parameters of the mathematical model. The system model in many cases is therefore not fully characterised either in terms of structure or the values which parameters take. This uncertainty must be accounted for in a systematic manner when the model is used in simulation or predictive mode to safeguard against reaching conclusions about system characteristics that are unwarranted, or in making predictions that are unjustifiably optimistic given the uncertainty about the model. The Bayesian inferential methodology provides a coherent framework with which to characterise and propagate uncertainty in such mechanistic models and this paper provides an introduction to Bayesian methodology as applied to system models represented as differential equations.     

Simulation-based Bayesian inference for epidemic models

A powerful and flexible method for fitting dynamic models to missing and censored data is to use the Bayesian paradigm via data-augmented Markov chain Monte Carlo (DA-MCMC). This samples from the joint posterior for the parameters and missing data, but requires high memory overheads for large-scale systems. In addition, designing efficient proposal distributions for the missing data is typically challenging. Pseudo-marginal methods instead integrate across the missing data using a Monte Carlo estimate for the likelihood, generated from multiple independent simulations from the model. These techniques can avoid the high memory requirements of DA-MCMC, and under certain conditions produce the exact marginal posterior distribution for parameters. A novel method is presented for implementing importance sampling for dynamic epidemic models, by conditioning the simulations on sets of validity criteria (based on the model structure) as well as the observed data. The flexibility of these techniques is illustrated using both removal time and final size data from an outbreak of smallpox. It is shown that these approaches can circumvent the need for reversible-jump MCMC, and can allow inference in situations where DA-MCMC is impossible due to computationally infeasible likelihoods.     

Bayesian inference for wind field retrieval

In many problems in spatial statistics it is necessary to infer a global problem solution by combining local models. A principled approach to this problem is to develop a global probabilistic model for the relationships between local variables and to use this as the prior in a Bayesian inference procedure. We use a Gaussian process with hyper-parameters estimated from numerical weather prediction models, which yields meteorologically convincing wind fields. We use neural networks to make local estimates of wind vector probabilities. The resulting inference problem cannot be solved analytically, but Markov Chain Monte Carlo methods allow us to retrieve accurate wind fields.     

Robust fuzzy CPU utilization control for dynamic workloads

In a number of real-time applications such as target tracking, precise workloads are unknown a priori but may dynamically vary, for example, based on the changing number of targets to track. It is important to manage the CPU utilization, via feedback control, to avoid severe overload or underutilization even in the presence of dynamic workloads. However, it is challenge to model a real-time system for feedback control, as computer systems cannot be modeled via physics laws. In this paper, we present a novel closed-loop approach for utilization control based on formal fuzzy logic control theory, which is very effective to support the desired performance in a nonlinear dynamic system without requiring a system model. We mathematically prove the stability of the fuzzy closed-loop system. Further, in a real-time kernel, we implement and evaluate our fuzzy logic utilization controller as well as two existing utilization controllers based on the linear and model predictive control theory for an extensive set of workloads. Our approach supports the specified average utilization set-point, while showing the best transient performance in terms of utilization control among the tested approaches.     

Structure inference for Bayesian multisensory scene understanding

We investigate a solution to the problem of multi-sensor scene understanding by formulating it in the framework of Bayesian model selection and structure inference. Humans robustly associate multimodal data as appropriate, but previous modelling work has focused largely on optimal fusion, leaving segregation unaccounted for and unexploited by machine perception systems. We illustrate a unifying, Bayesian solution to multi-sensor perception and tracking which accounts for both integration and segregation by explicit probabilistic reasoning about data association in a temporal context. Such explicit inference of multimodal data association is also of intrinsic interest for higher level understanding of multisensory data. We illustrate this using a probabilistic implementation of data association in a multi-party audio-visual scenario, where unsupervised learning and structure inference is used to automatically segment, associate and track individual subjects in audiovisual sequences. Indeed, the structure inference based framework introduced in this work provides the theoretical foundation needed to satisfactorily explain many confounding results in human psychophysics experiments involving multimodal cue integration and association.     

CPU/GPU异构计算应用于核电模拟机的可行性

根据ANSI/ANS-3.5-1998规定以及核电厂建模精度的提高,对核电模拟机仿真速度提出了更高的要求。但是目前已难以通过提升中央处理器（ CPU）频率的方式来提升现有模拟机的运算速度。与此同时,CPU/GPU异构计算融合了串行/并行计算,利用显卡（ GPU）的并行计算能力可极大提升现有桌面电脑的运算能力,目前已经广泛应用于科学研究。英伟达公司的CUDA平台被用于开发CPU/GPU异构计算应用程序,来提升核电厂全范围模拟机的仿真计算。通过核电厂全范围模拟机运行测试对比,证实使用CPU/GPU异构计算程序,能有效提升模拟机运行速度。     

Mathematical programming models for scheduling in a CPU/FPGA architecture with heterogeneous communication delays

This paper deals with the mathematical modelling of a scheduling problem in a heterogeneous CPU/FPGA architecture with heterogeneous communication delays in order to minimize the makespan, \(C_{max}\). This study was motivated by the quality of the available solvers for Mixed Integer Program. The proposed model includes the communication delay constraints in a heterogeneous case, depending on both tasks and computing units. These constraints are linearized without adding any extra variables and the obtained linear model is reduced to speed-up the solving with CPLEX up to 60 times. Computational results show that the proposed model is promising. For an average sized problem of up to 50 tasks and five computing units the solving time under CPLEX is a few seconds.     

Parallel patterns for heterogeneous CPU/GPU architectures: Structured parallelism from cluster to cloud

The widespread adoption of traditional heterogeneous systems has substantially improved the computing power available and, in the meantime, raised optimisation issues related to the processing of task streams across both CPU and GPU cores in heterogeneous systems. Similar to the heterogeneous improvement gained in traditional systems, cloud computing has started to add heterogeneity support, typically through GPU instances, to the conventional CPU-based cloud resources. This optimisation of cloud resources will arguably have a real impact when running on-demand computationally-intensive applications.In this work, we investigate the scaling of pattern-based parallel applications from physical, “local” mixed CPU/GPU-clusters to a public cloud CPU/GPU infrastructure. Specifically, such parallel patterns are deployed via algorithmic skeletons to exploit a peculiar parallel behaviour while hiding implementation details.We propose a systematic methodology to exploit approximated analytical performance/cost models, and an integrated programming framework that is suitable for targeting both local and remote resources to support the offloading of computations from structured parallel applications to heterogeneous cloud resources, such that performance values not available on local resources may be actually achieved with the remote resources. The amount of remote resources necessary to achieve a given performance target is calculated through the performance models in order to allow any user to hire the amount of cloud resources needed to achieve a given target performance value. Thus, it is therefore expected that such models can be used to devise the optimal proportion of computations to be allocated on different remote nodes for Big Data computations.We present different experiments run with a proof-of-concept implementation based on FastFlow  on small departmental clusters as well as on a public cloud infrastructure with CPU and GPU using the Amazon Elastic Compute Cloud. In particular, we show how CPU-only and mixed CPU/GPU computations can be offloaded to remote cloud resources with predictable performances and how data intensive applications can be mapped to a mix of local and remote resources to guarantee optimal performances.     

Bayesian inference for additive mixed quantile regression models

Quantile regression problems in practice may require flexible semiparametric forms of the predictor for modeling the dependence of responses on covariates. Furthermore, it is often necessary to add random effects accounting for overdispersion caused by unobserved heterogeneity or for correlation in longitudinal data. We present a unified approach for Bayesian quantile inference on continuous response via Markov chain Monte Carlo (MCMC) simulation and approximate inference using integrated nested Laplace approximations (INLA) in additive mixed models. Different types of covariate are all treated within the same general framework by assigning appropriate Gaussian Markov random field (GMRF) priors with different forms and degrees of smoothness. We applied the approach to extensive simulation studies and a Munich rental dataset, showing that the methods are also computationally efficient in problems with many covariates and large datasets.     

Online Bayesian inference for the parameters of PRISM programs

This paper presents a method for approximating posterior distributions over the parameters of a given PRISM program. A sequential approach is taken where the distribution is updated one datapoint at a time. This makes it applicable to online learning situations where data arrives over time. The method is applicable whenever the prior is a mixture of products of Dirichlet distributions. In this case the true posterior will be a mixture of very many such products. An approximation is effected by merging products of Dirichlet distributions. An analysis of the quality of the approximation is presented. Due to the heavy computational burden of this approach, the method has been implemented in the Mercury logic programming language. Initial results using a hidden Markov model and a probabilistic graph are presented.     

Mobile context inference using two-layered Bayesian networks for smartphones

Recently, mobile context inference becomes an important issue. Bayesian probabilistic model is one of the most popular probabilistic approaches for context inference. It efficiently represents and exploits the conditional independence of propositions. However, there are some limitations for probabilistic context inference in mobile devices. Mobile devices relatively lacks of sufficient memory. In this paper, we present a novel method for efficient Bayesian inference on a mobile phone. In order to overcome the constraints of the mobile environment, the method uses two-layered Bayesian networks with tree structure. In contrast to the conventional techniques, this method attempts to use probabilistic models with fixed tree structures and intermediate nodes. It can reduce the inference time by eliminating junction tree creation. To evaluate the performance of this method, an experiment is conducted with data collected over a month. The result shows the efficiency and effectiveness of the proposed method.     

Mean-field variational approximate Bayesian inference for latent variable models

The ill-posed nature of missing variable models offers a challenging testing ground for new computational techniques. This is the case for the mean-field variational Bayesian inference. The behavior of this approach in the setting of the Bayesian probit model is illustrated. It is shown that the mean-field variational method always underestimates the posterior variance and, that, for small sample sizes, the mean-field variational approximation to the posterior location could be poor.     

Pseudo-marginal Bayesian inference for Gaussian process latent variable models

Machine Learning - A Bayesian inference framework for supervised Gaussian process latent variable models is introduced. The framework overcomes the high correlations between latent variables and...     

Vector performance estimation for CRAY X-MP/Y-MP supercomputers

Optimization of vector-intensive applications for the CRAY X-MP/Y-MP often requires arranging the operations to take full advantage of such architectural features as the memory system, independent memory ports, chaining, and independent functional units. Estimation of performance is not straightforward since many operations can occur concurrently. As a tool for making trades between vector algorithms, a method has been developed and used successfully at E-Systems Inc. to predict the execution time of a sequence of vector operations without resorting to actual code development. This method reduced our software development time, produced significantly more efficient code, and provided for a systematic approach to optimization. The performance estimation is generally accurate to within 10% and accounts for memory conflicts that result from fixed stride references.     

Hierarchical CPU utilization control for real-time guarantees in power grid computing

Blackouts in our daily life can be disastrous with enormous economic loss. Blackouts usually occur when appropriate corrective actions are not effectively taken for an initial contingency. Therefore, it is critical to complete those tasks that are running power grid computing algorithms in the Energy Management System (EMS) in a timely manner to avoid blackouts. This problem can be formulated as guaranteeing end-to-end deadlines in a Distributed Real-time Embedded (DRE) system. However, existing feedback scheduling algorithms in DRE systems cannot be directly adopted to handle with significantly different timescales of power grid computing tasks. In this paper, we propose a hierarchical control solution to guarantee the deadlines of those tasks in EMS by grouping them based on their characteristics. Furthermore, we present an adaptive control scheme to achieve analytical assurance of control accuracy and system stability, in spite of significant system variation. Our solution is based on well-established control theory for guaranteed control accuracy and system stability and can adapt to changes in the system model without manual reconfiguration and profiling. Simulation results based on a realistic workload configuration demonstrate that our solution can guarantee timeliness for power grid computing and hence help to avoid blackouts.     

Bayesian inference for an item response model for modeling test anxiety

We develop a Bayesian binary Item Response Model (IRM), which we denote as Test Anxiety Model (TAM), for estimating the proficiency scores when individuals might experience test anxiety. We consider order restricted item parameters conditionally to the examinees’ reported emotional state at the testing session. We consider three test anxiety levels: calm, anxious and very anxious. Using simulated data we show that taking into account test anxiety levels in an IRM help us to obtain fair proficiency estimates as opposed to the ones obtained with three two-parameter logistic IRM (3PM) by Birnbaum (1957, 1968). For the 3PM, the proficiency estimates tend to be positively biased for both, calm and anxious examinees.     

Approximate inference for dynamic Bayesian networks: sliding window approach

Dynamic Bayesian networks (DBNs) are probabilistic graphical models that have become a ubiquitous tool for compactly describing statistical relationships among a group of stochastic processes. A suite of elaborately designed inference algorithms makes it possible for intelligent systems to use a DBN to make inferences in uncertain conditions. Unfortunately, exact inference or even approximation in a DBN has been proved to be NP-hard and is generally computationally prohibitive. In this paper, we investigate a sliding window framework for approximate inference in DBNs to reduce the computational burden. By introducing a sliding window that moves forward as time progresses, inference at any time is restricted to a quite narrow region of the network. The main contributions to the sliding window framework include an exploration of its foundations, explication of how it operates, and the proposal of two strategies for adaptive window size selection. To make this framework available as an inference engine, the interface algorithm widely used in exact inference is then integrated with the framework for approximate inference in DBNs. After analyzing its computational complexity, further empirical work is presented to demonstrate the validity of the proposed algorithms.     

二值probit回归模型的坍缩变分贝叶斯推断算法

给出了二值probit回归模型的坍缩变分贝叶斯推断算法.此算法比变分贝叶斯推断算法能更逼近对数边缘似然,得到更精确的模型参数后验期望值.如果两个算法得到的分类错误一致,则该算法的迭代次数较变分法明显减少.仿真实验结果验证了所提出算法的有效性.