Volume 4, Issue 1
A Splitting Hamiltonian Monte Carlo Method for Efficient Sampling

Lei Li, Lin Liu & Yuzhou Peng

CSIAM Trans. Appl. Math., 4 (2023), pp. 41-73.

Published online: 2023-01

Export citation
  • Abstract

We propose a splitting Hamiltonian Monte Carlo (SHMC) algorithm, which can be computationally efficient when combined with the random mini-batch strategy. By splitting the potential energy into numerically nonstiff and stiff parts, one makes a proposal using the nonstiff part of $U,$ followed by a Metropolis rejection step using the stiff part that is often easy to compute. The splitting allows efficient sampling from systems with singular potentials (or distributions with degenerate points) and/or with multiple potential barriers. In our SHMC algorithm, the proposal only based on the nonstiff part in the splitting is generated by the Hamiltonian dynamics, which can be potentially more efficient than the overdamped Langevin dynamics. We also use random batch strategies to reduce the computational cost to $\mathcal{O}(1)$ per time step in generating the proposals for problems arising from many-body systems and Bayesian inference, and prove that the errors of the Hamiltonian induced by the random batch approximation is $\mathcal{O}(\sqrt{∆t})$ in the strong and $\mathcal{O}(∆t)$ in the weak sense, where $∆t$ is the time step. Numerical experiments are conducted to verify the theoretical results and the computational efficiency of the proposed algorithms in practice.

  • AMS Subject Headings

62M05, 65C05

  • Copyright

COPYRIGHT: © Global Science Press

  • Email address
  • BibTex
  • RIS
  • TXT
@Article{CSIAM-AM-4-41, author = {Li , LeiLiu , Lin and Peng , Yuzhou}, title = {A Splitting Hamiltonian Monte Carlo Method for Efficient Sampling}, journal = {CSIAM Transactions on Applied Mathematics}, year = {2023}, volume = {4}, number = {1}, pages = {41--73}, abstract = {

We propose a splitting Hamiltonian Monte Carlo (SHMC) algorithm, which can be computationally efficient when combined with the random mini-batch strategy. By splitting the potential energy into numerically nonstiff and stiff parts, one makes a proposal using the nonstiff part of $U,$ followed by a Metropolis rejection step using the stiff part that is often easy to compute. The splitting allows efficient sampling from systems with singular potentials (or distributions with degenerate points) and/or with multiple potential barriers. In our SHMC algorithm, the proposal only based on the nonstiff part in the splitting is generated by the Hamiltonian dynamics, which can be potentially more efficient than the overdamped Langevin dynamics. We also use random batch strategies to reduce the computational cost to $\mathcal{O}(1)$ per time step in generating the proposals for problems arising from many-body systems and Bayesian inference, and prove that the errors of the Hamiltonian induced by the random batch approximation is $\mathcal{O}(\sqrt{∆t})$ in the strong and $\mathcal{O}(∆t)$ in the weak sense, where $∆t$ is the time step. Numerical experiments are conducted to verify the theoretical results and the computational efficiency of the proposed algorithms in practice.

}, issn = {2708-0579}, doi = {https://doi.org/10.4208/csiam-am.SO-2021-0031}, url = {http://global-sci.org/intro/article_detail/csiam-am/21335.html} }
TY - JOUR T1 - A Splitting Hamiltonian Monte Carlo Method for Efficient Sampling AU - Li , Lei AU - Liu , Lin AU - Peng , Yuzhou JO - CSIAM Transactions on Applied Mathematics VL - 1 SP - 41 EP - 73 PY - 2023 DA - 2023/01 SN - 4 DO - http://doi.org/10.4208/csiam-am.SO-2021-0031 UR - https://global-sci.org/intro/article_detail/csiam-am/21335.html KW - Markov chain Monte Carlo, potential splitting, random-batch method, many body systems, Bayesian inference. AB -

We propose a splitting Hamiltonian Monte Carlo (SHMC) algorithm, which can be computationally efficient when combined with the random mini-batch strategy. By splitting the potential energy into numerically nonstiff and stiff parts, one makes a proposal using the nonstiff part of $U,$ followed by a Metropolis rejection step using the stiff part that is often easy to compute. The splitting allows efficient sampling from systems with singular potentials (or distributions with degenerate points) and/or with multiple potential barriers. In our SHMC algorithm, the proposal only based on the nonstiff part in the splitting is generated by the Hamiltonian dynamics, which can be potentially more efficient than the overdamped Langevin dynamics. We also use random batch strategies to reduce the computational cost to $\mathcal{O}(1)$ per time step in generating the proposals for problems arising from many-body systems and Bayesian inference, and prove that the errors of the Hamiltonian induced by the random batch approximation is $\mathcal{O}(\sqrt{∆t})$ in the strong and $\mathcal{O}(∆t)$ in the weak sense, where $∆t$ is the time step. Numerical experiments are conducted to verify the theoretical results and the computational efficiency of the proposed algorithms in practice.

Lei Li, Lin Liu & Yuzhou Peng. (2023). A Splitting Hamiltonian Monte Carlo Method for Efficient Sampling. CSIAM Transactions on Applied Mathematics. 4 (1). 41-73. doi:10.4208/csiam-am.SO-2021-0031
Copy to clipboard
The citation has been copied to your clipboard