Title:The Block Pseudo-Marginal Sampler

Abstract:The pseudo-marginal (PM) approach is used increasingly for Bayesian inference in statistical models where the likelihood is intractable but can be estimated unbiasedly. Examples include random effect models, state-space models and data subsampling in big-data settings. Deligiannidis et al (2016) show how the PM approach can be made much more efficient by correlating the underlying Monte Carlo (MC) random numbers used to form the estimate of the likelihood at the current and proposed values of the unknown parameters. Their approach greatly speeds up the standard PM algorithm, as it requires a much smaller number of samples or particles to form the optimal likelihood estimate. We present an alternative PM approachthat divides the underlying random numbers into blocks so that the likelihood estimates for the proposed and current values of the parameters only differ by the random numbers in one block. For a number of important applications, our approach has several advantages over that of Deligiannidis et al (2016). First, is a natural way to carry out PM in many problems, e.g., when the likelihood is a product of integrals and is estimated by randomised quasi-Monte Carlo, we show that the optimal number of samples required in the block PM is much smaller than that in~Deligiannidis et al (2016). Second, the block PM provides a direct way to control the correlation between the logarithms of the estimates of the likelihood at the current and proposed values of the parameters, whereas the correlated PM may not. We obtain theory and guidelines for selecting the optimal number of samples, and document large speed-ups in a wide range of applications.