TY - JOUR
T1 - Split Hamiltonian Monte Carlo
AU - Shahbaba, Babak
AU - Lan, Shiwei
AU - Johnson, Wesley O.
AU - Neal, Radford M.
N1 - Funding Information:
Acknowledgements B. Shahbaba is supported by the National Science Foundation, Grant No. IIS-1216045. R.M. Neal’s work is sup- ported by the Natural Sciences and Engineering Research Council of Canada. He holds a Canada Research Chair in Statistics and Machine Learning.
PY - 2014/5
Y1 - 2014/5
N2 - We show how the Hamiltonian Monte Carlo algorithm can sometimes be speeded up by "splitting" the Hamiltonian in a way that allows much of the movement around the state space to be done at low computational cost. One context where this is possible is when the log density of the distribution of interest (the potential energy function) can be written as the log of a Gaussian density, which is a quadratic function, plus a slowly-varying function. Hamiltonian dynamics for quadratic energy functions can be analytically solved. With the splitting technique, only the slowly-varying part of the energy needs to be handled numerically, and this can be done with a larger stepsize (and hence fewer steps) than would be necessary with a direct simulation of the dynamics. Another context where splitting helps is when the most important terms of the potential energy function and its gradient can be evaluated quickly, with only a slowly-varying part requiring costly computations. With splitting, the quick portion can be handled with a small stepsize, while the costly portion uses a larger stepsize. We show that both of these splitting approaches can reduce the computational cost of sampling from the posterior distribution for a logistic regression model, using either a Gaussian approximation centered on the posterior mode, or a Hamiltonian split into a term that depends on only a small number of critical cases, and another term that involves the larger number of cases whose influence on the posterior distribution is small.
AB - We show how the Hamiltonian Monte Carlo algorithm can sometimes be speeded up by "splitting" the Hamiltonian in a way that allows much of the movement around the state space to be done at low computational cost. One context where this is possible is when the log density of the distribution of interest (the potential energy function) can be written as the log of a Gaussian density, which is a quadratic function, plus a slowly-varying function. Hamiltonian dynamics for quadratic energy functions can be analytically solved. With the splitting technique, only the slowly-varying part of the energy needs to be handled numerically, and this can be done with a larger stepsize (and hence fewer steps) than would be necessary with a direct simulation of the dynamics. Another context where splitting helps is when the most important terms of the potential energy function and its gradient can be evaluated quickly, with only a slowly-varying part requiring costly computations. With splitting, the quick portion can be handled with a small stepsize, while the costly portion uses a larger stepsize. We show that both of these splitting approaches can reduce the computational cost of sampling from the posterior distribution for a logistic regression model, using either a Gaussian approximation centered on the posterior mode, or a Hamiltonian split into a term that depends on only a small number of critical cases, and another term that involves the larger number of cases whose influence on the posterior distribution is small.
KW - Bayesian analysis
KW - Hamiltonian dynamics
KW - Markov chain Monte Carlo
UR - http://www.scopus.com/inward/record.url?scp=84898540704&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=84898540704&partnerID=8YFLogxK
U2 - 10.1007/s11222-012-9373-1
DO - 10.1007/s11222-012-9373-1
M3 - Article
AN - SCOPUS:84898540704
SN - 0960-3174
VL - 24
SP - 339
EP - 349
JO - Statistics and Computing
JF - Statistics and Computing
IS - 3
ER -