Markov Chain Monte Carlo

Gist

Used for Bayesian analysis, how to simulate the posterior distribution

Definiton

Metropolis-Hastings

Algorithm

Step 1: Propose a new location.
Conditioned on the current location $\mu$, draw a location ${\mu }^{\prime }$ from a proposal model with pdf $q({\mu }^{\prime }|\mu$).

Step 2: Decide whether or not to go there

• Calculate the acceptance probability ${\mu }^{\prime }$:
  $$\alpha = min(1, \frac{f(\mu') L(\mu'|y)(q(\mu|\mu')}{f(\mu') L(\mu'|y)(q(\mu|\mu')}$$
• Figuratively, flip a weighted coin. If it's Heads, with probability $\alpha$, go to the proposed location ${\mu }^{\prime }$. If it's Tails, with probability $1-\alpha$, stay at $\mu$.

Metropolis algorithm

The Metropolis algorithm is a special case of the Metropolis Hastings in which the proposal model is symmetric.

Step 1: Propose a random location, u', for the next tour stop
Step 2: Decide whether to go to the proposed location ($u^{t-1}=u^{\prime }$ ) or to stay at the current location for another iteration ($u^{t+1}=u$)

We can propose Markov chain tour stops by sampling from a more different model such as a uniform proposal model. If the proposed location is greater than that of the current location- definitely go there. Otherwise, maybe go there?

Diagnostic

Trace plot

An example of a good trace-plot

Density plots of parallel chains

Effective sample ratio

Autocorrelation

See Autocorrelation

R-hat

Calculates the ratio between sources of variability - across all chains combined and then variability within any individual chain.

If R-hat = 1, this reflects stability. In contrast, R-hat > 1 indicates instability. R-hat ratio greater than 1.05 raises some red flags about the stability of the simulation

Program

Stan uses Hamiltonian Monte Carlo
Jags uses Gibb sampling