Distributed computing by the Monte Carlo method

Questions of effective parallel realization for some algorithms of the Monte Carlo method are discussed. Parallel modification of the generator of basic pseudorandom numbers uniformly distributed in the unit interval is described. The technique of distributed computing in the personal computer network with the use of the MONC program system worked out by the authors is described.     

An evolutionary Monte Carlo algorithm for Bayesian block clustering of data matrices

In many applications, it is of interest to simultaneously cluster row and column variables in a data set, identifying local subgroups within a data matrix that share some common characteristic. When a small set of variables is believed to be associated with a set of responses, block clustering or biclustering is a more appropriate technique to use compared to one-dimensional clustering. A flexible framework for Bayesian model-based block clustering, that can determine multiple block clusters in a data matrix through a novel and efficient evolutionary Monte Carlo-based methodology, is proposed. The performance of this methodology is illustrated through a number of simulation studies and an application to data from genome-wide association studies.     

Bayesian Collaborative Denoising for Monte Carlo Rendering

The stochastic nature of Monte Carlo rendering algorithms inherently produces noisy images. Essentially, three approaches have been developed to solve this issue: improving the ray‐tracing strategies to reduce pixel variance, providing adaptive sampling by increasing the number of rays in regions needing so, and filtering the noisy image as a post‐process. Although the algorithms from the latter category introduce bias, they remain highly attractive as they quickly improve the visual quality of the images, are compatible with all sorts of rendering effects, have a low computational cost and, for some of them, avoid deep modifications of the rendering engine. In this paper, we build upon recent advances in both non‐local and collaborative filtering methods to propose a new efficient denoising operator for Monte Carlo rendering. Starting from the local statistics which emanate from the pixels sample distribution, we enrich the image with local covariance measures and introduce a nonlocal bayesian filter which is specifically designed to address the noise stemming from Monte Carlo rendering. The resulting algorithm only requires the rendering engine to provide for each pixel a histogram and a covariance matrix of its color samples. Compared to state‐of‐the‐art sample‐based methods, we obtain improved denoising results, especially in dark areas, with a large increase in speed and more robustness with respect to the main parameter of the algorithm. We provide a detailed mathematical exposition of our bayesian approach, discuss extensions to multiscale execution, adaptive sampling and animated scenes, and experimentally validate it on a collection of scenes.     

Sequential Monte Carlo for Bayesian matching of objects with occlusions

We consider the problem of locating instances of a known object in a novel scene by matching the fiducial features of the object. The appearance of the features and the shape of the object are modeled separately and combined in a Bayesian framework. In this paper, we present a novel matching scheme based on sequential Monte Carlo, in which the features are matched sequentially, utilizing the information about the locations of previously matched features to constrain the task. The particle representation of hypotheses about the object position allow matching in multimodal and cluttered environments, where batch algorithms may have convergence difficulties. The proposed method requires no initialization or predetermined matching order, as the sequence can be started from any feature. We also utilize a Bayesian model to deal with features that are not detected due to occlusions or abnormal appearance. In our experiments, the proposed matching system shows promising results, with performance equal to batch approaches when the target distribution is unimodal, while surpassing traditional methods under multimodal conditions. Using the occlusion model, the object can be localized from only a few visible features, with the nonvisible parts predicted from the conditional prior model.     

A Bayesian Monte Carlo Approach to Global Illumination

Most Monte Carlo rendering algorithms rely on importance sampling to reduce the variance of estimates. Importance sampling is efficient when the proposal sample distribution is well‐suited to the form of the integrand but fails otherwise. The main reason is that the sample location information is not exploited. All sample values are given the same importance regardless of their proximity to one another. Two samples falling in a similar location will have equal importance whereas they are likely to contain redundant information. The Bayesian approach we propose in this paper uses both the location and value of the data to infer an integral value based on a prior probabilistic model of the integrand. The Bayesian estimate depends only on the sample values and locations, and not how these samples have been chosen. We show how this theory can be applied to the final gathering problem and present results that clearly demonstrate the benefits of Bayesian Monte Carlo.     

Monte Carlo methods for computing the capacitance of the unit cube

It is well known that there is no analytic expression for the electrical capacitance of the unit cube. However, there are several Monte Carlo methods that have been used to numerically estimate this capacitance to high accuracy. These include a Brownian dynamics algorithm [H.-X. Zhou, A. Szabo, J.F. Douglas, J.B. Hubbard, A Brownian dynamics algorithm for calculating the hydrodynamic friction and the electrostatic capacitance of an arbitrarily shaped object, J. Chem. Phys. 100 (5) (1994) 3821–3826] coupled to the “walk on spheres” (WOS) method [M.E. Müller, Some continuous Monte Carlo methods for the Dirichlet problem, Ann. Math. Stat. 27 (1956) 569–589]; the Green’s function first-passage (GFFP) algorithm [J.A. Given, J.B. Hubbard, J.F. Douglas, A first-passage algorithm for the hydrodynamic friction and diffusion-limited reaction rate of macromolecules, J. Chem. Phys. 106 (9) (1997) 3721–3771]; an error-controlling Brownian dynamics algorithm [C.-O. Hwang, M. Mascagni, Capacitance of the unit cube, J. Korean Phys. Soc. 42 (2003) L1–L4]; an extrapolation technique coupled to the WOS method [C.-O. Hwang, Extrapolation technique in the “walk on spheres” method for the capacitance of the unit cube, J. Korean Phys. Soc. 44 (2) (2004) 469–470]; the “walk on planes” (WOP) method [M.L. Mansfield, J.F. Douglas, E.J. Garboczi, Intrinsic viscosity and the electrical polarizability of arbitrarily shaped objects, Phys. Rev. E 64 (6) (2001) 061401:1–061401:16; C.-O. Hwang, M. Mascagni, Electrical capacitance of the unit cube, J. Appl. Phys. 95 (7) (2004) 3798–3802]; and the random “walk on the boundary” (WOB) method [M. Mascagni, N.A. Simonov, The random walk on the boundary method for calculating capacitance, J. Comp. Phys. 195 (2004) 465–473]. Monte Carlo methods are convenient and efficient for problems whose solution includes singularities. In the calculation of the unit cube capacitance, there are edge and corner singularities in the charge density distribution. In this paper, we review the above Monte Carlo methods for computing the electrical capacitance of a cube and compare their effectiveness. We also provide a new result. We will focus our attention particularly on two Monte Carlo methods: WOP [M.L. Mansfield, J.F. Douglas, E.J. Garboczi, Intrinsic viscosity and the electrical polarizability of arbitrarily shaped objects, Phys. Rev. E 64 (6) (2001) 061401:1–061401:16; C.-O. Hwang, M. Mascagni, Electrical capacitance of the unit cube, J. Appl. Phys. 95 (7) (2004) 3798–3802; C.-O. Hwang, T. Won, Edge charge singularity of conductors, J. Korean Phys. Soc. 45 (2004) S551–S553] and the random WOB [M. Mascagni, N.A. Simonov, The random walk on the boundary method for calculating capacitance, J. Comp. Phys. 195 (2004) 465–473] methods.     

Exchange Monte Carlo Sampling From Bayesian Posterior for Singular Learning Machines

Many singular learning machines such as neural networks, normal mixtures, Bayesian networks, and hidden Markov models belong to singular learning machines and are widely used in practical information systems. In these learning machines, it is well known that Bayesian learning provides better generalization performance than maximum-likelihood estimation. However, it needs huge computational cost to sample from a Bayesian posterior distribution of a singular learning machine by a conventional Markov chain Monte Carlo (MCMC) method, such as the metropolis algorithm, because of singularities. Recently, the exchange Monte Carlo (MC) method, which is well known as an improved algorithm of MCMC method, has been proposed to apply to Bayesian neural network learning in the literature. In this paper, we propose the idea that the exchange MC method has a better effect on Bayesian learning in singular learning machines than that in regular learning machines, and show its effectiveness by comparing the numerical stochastic complexity with the theoretical one.      

A Sequential Monte Carlo Method for Bayesian Analysis of Massive Datasets

Markov chain Monte Carlo (MCMC) techniques revolutionized statistical practice in the 1990s by providing an essential toolkit for making the rigor and flexibility of Bayesian analysis computationally practical. At the same time the increasing prevalence of massive datasets and the expansion of the field of data mining has created the need for statistically sound methods that scale to these large problems. Except for the most trivial examples, current MCMC methods require a complete scan of the dataset for each iteration eliminating their candidacy as feasible data mining techniques.In this article we present a method for making Bayesian analysis of massive datasets computationally feasible. The algorithm simulates from a posterior distribution that conditions on a smaller, more manageable portion of the dataset. The remainder of the dataset may be incorporated by reweighting the initial draws using importance sampling. Computation of the importance weights requires a single scan of the remaining observations. While importance sampling increases efficiency in data access, it comes at the expense of estimation efficiency. A simple modification, based on the rejuvenation step used in particle filters for dynamic systems models, sidesteps the loss of efficiency with only a slight increase in the number of data accesses.To show proof-of-concept, we demonstrate the method on two examples. The first is a mixture of transition models that has been used to model web traffic and robotics. For this example we show that estimation efficiency is not affected while offering a 99% reduction in data accesses. The second example applies the method to Bayesian logistic regression and yields a 98% reduction in data accesses.     

Stochastic gradient Hamiltonian Monte Carlo with variance reduction for Bayesian inference

Machine Learning - Gradient-based Monte Carlo sampling algorithms, like Langevin dynamics and Hamiltonian Monte Carlo, are important methods for Bayesian inference. In large-scale settings,...     

Bayesian system identification via Markov chain Monte Carlo techniques

The work here explores new numerical methods for supporting a Bayesian approach to parameter estimation of dynamic systems. This is primarily motivated by the goal of providing accurate quantification of estimation error that is valid for arbitrary, and hence even very short length data records. The main innovation is the employment of the Metropolis-Hastings algorithm to construct an ergodic Markov chain with invariant density equal to the required posterior density. Monte Carlo analysis of samples from this chain then provides a means for efficiently and accurately computing posteriors for model parameters and arbitrary functions of them.     

Parallel computing for lattice Monte Carlo simulation of large-scale thin film growth

This paper proposes two viable computing strategies for distributed parallel systems: domain division with sub-domain overlapping and asynchronous communication. We have implemented a parallel computing procedure for simulation of Ti thin film growing process of a system with 1000 x 1000 atoms by means of the Monte Carlo (MC) method. This approach greatly reduces the computation time for simulation of large-scale thin film growth under realistic deposition rates. The multi-lattice MC model of deposition comprises two basic events: deposition, and surface diffusion. Since diffusion constitutes more than 90% of the total simulation time of the whole deposition process at high temperature, we concentrated on implementing a new parallel diffusion simulation that reduces communication time during simulation. Asynchronous communication and domain overlapping techniques are used to reduce the waiting time and communication time among parallel processors. The parallel algorithms we propose can simulate the thin     

Distributed implementation of the adaptive kinetic Monte Carlo method

The program EON2 is a distributed implementation of the adaptive kinetic Monte Carlo method for long time scale simulations of atomistic systems. The method is based on the transition state theory approach within the harmonic approximation and the key step is the identification of relevant saddle points on the potential energy rim surrounding the energy minimum corresponding to a state of the system. The saddle point searches are carried out in a distributed fashion starting with random initial displacements of the atoms in regions where atoms have less than optimal coordination. The main priorities of this implementation have been to (1) make the code transparent, (2) decouple the master and slaves, and (3) have a well defined interface to the energy and force evaluation. The computationally intensive parts are implemented in C++, whereas the less compute intensive server-side software is written in Python. The platform for distributed computing is BOINC. A simulation of the annealing of a twist and tilt grain boundary in a copper crystal is described as an example application.     

A Beowulf-class computing cluster for the Monte Carlo production of the LHCb experiment

The computing cluster built at Bologna to provide the LHCb Collaboration with a powerful Monte Carlo production tool is presented. It is a performance oriented Beowulf-class cluster, made of rack mounted commodity components, designed to minimize operational support requirements and to provide full and continuous availability of the computing resources. In this paper we describe the architecture of the cluster, and discuss the technical solutions adopted for each specialized sub-system.     

Optimal computing budget allocation for Monte Carlo simulation with application to product design

Ordinal optimization has emerged as an efficient technique for simulation and optimization, converging exponentially in many cases. In this paper, we present a new computing budget allocation approach that further enhances the efficiency of ordinal optimization. Our approach intelligently determines the best allocation of simulation trials or samples necessary to maximize the probability of identifying the optimal ordinal solution. We illustrate the approach’s benefits and ease of use by applying it to two electronic circuit design problems. Numerical results indicate the approach yields significant savings in computation time above and beyond the use of ordinal optimization.     

Distributed Markov Chain Monte Carlo kernel based particle filtering for object tracking

Particle filters are computationally intensive and thus efficient parallelism is crucial to effective implementations, especially object tracking in video sequences. Two schemes for pipelining particles under high performance computing environment, including an alternative Markov Chain Monte Carlo (MCMC) resampling algorithm and kernel function, are proposed so as to improve tracking performance and minimize execution time. Experimental results on a network of workstations composed of simple off-the-shelf hardware components show that global parallelizable scheme provides a promising resolution to clearly reduce execution time with increasing particles, compared with generic particle filtering.     

Overview of Bayesian sequential Monte Carlo methods for group and extended object tracking

This work presents the current state-of-the-art in techniques for tracking a number of objects moving in a coordinated and interacting fashion. Groups are structured objects characterized with particular motion patterns. The group can be comprised of a small number of interacting objects (e.g. pedestrians, sport players, convoy of cars) or of hundreds or thousands of components such as crowds of people. The group object tracking is closely linked with extended object tracking but at the same time has particular features which differentiate it from extended objects. Extended objects, such as in maritime surveillance, are characterized by their kinematic states and their size or volume. Both group and extended objects give rise to a varying number of measurements and require trajectory maintenance. An emphasis is given here to sequential Monte Carlo (SMC) methods and their variants. Methods for small groups and for large groups are presented, including Markov Chain Monte Carlo (MCMC) methods, the random matrices approach and Random Finite Set Statistics methods. Efficient real-time implementations are discussed which are able to deal with the high dimensionality and provide high accuracy. Future trends and avenues are traced.     

New hybrid Monte Carlo methods and computing the dominant generalized eigenvalue

In this paper, the computation of dominant generalized eigenvalue problem using the new Monte Carlo method are presented. We also compare the numerical results and the CPU-time of two different methods for evaluating dominant generalized eigenvalue. The first method is the QR method. The second method is the extended Monte Carlo method which is called the resolvent Monte Carlo algorithm. Finally, using these methods the numerical results for the general symmetric dense/sparse matrices are performed.     

Multi-objective evolutionary approach to prevent premature convergence in Monte Carlo localization

In this paper, we propose a global localization algorithm for mobile robots based on Monte Carlo localization (MCL), which employs multi-objective particle swarm optimization (MOPSO) incorporating a novel archiving strategy, to deal with the premature convergence problem in global localization in highly symmetrical environments. Under three proposed rules, premature convergence occurring during the localization can be easily detected so that the proposed MOPSO is introduced to obtain a uniformly distributed Pareto front based on two objective functions respectively representing weights and distribution of particles in MCL. On the basis of the derived Pareto front, MCL is able to resample particles with balanced weights as well as diverse distribution of the population. As a consequence, the proposed approach provides better diversity for particles to explore the environment, while simultaneously maintaining good convergence to achieve a successful global localization. Simulations have confirmed that the proposed approach can significantly improve global localization performance in terms of success rate and computational time in highly symmetrical environments.