What is MCMC and what methods are there (Metropolis-Hastings, Gibbs, HMC)?

MCMC (Markov Chain Monte Carlo) is a class of algorithms for generating samples from complex distributions, especially when direct computation is impossible. It constructs a Markov chain that converges to the target distribution over time.

Main MCMC methods:

• Metropolis-Hastings: a basic algorithm that generates candidates for the next state and accepts them with a certain probability, ensuring convergence to the desired distribution.

• Gibbs Sampling: a special case of Metropolis-Hastings where each parameter is updated sequentially from the conditional distribution given the other parameters. It is particularly effective if the conditional distributions are known and simple.

• Hamiltonian Monte Carlo (HMC): uses gradients of the log of the target distribution and the physical interpretation of Hamiltonian dynamics for more efficient exploration of the parameter space, reducing random walks and speeding up convergence.

These methods are widely used in Bayesian inference, Gaussian Processes, and Bayesian Optimization for estimating complex posterior distributions.