Markov Chain Monte Carlo

Gist

Used for Bayesian analysis, how to simulate the posterior distribution

Definition

Metropolis-Hastings

Algorithm

Step 1: Propose a new location.
Conditioned on the current location $\mu$, draw a location ${\mu }^{\prime }$ from a proposal model with pdf $q({\mu }^{\prime }|\mu$).

Step 2: Decide whether or not to go there

• Calculate the acceptance probability ${\mu }^{\prime }$:
  $$\alpha = min(1, \frac{f(\mu') L(\mu'|y)(q(\mu|\mu')}{f(\mu') L(\mu'|y)(q(\mu|\mu')}$$
• Figuratively, flip a weighted coin. If it's Heads, with probability $\alpha$, go to the proposed location ${\mu }^{\prime }$. If it's Tails, with probability $1-\alpha$, stay at $\mu$.

Metropolis algorithm

The Metropolis algorithm is a special case of the Metropolis Hastings in which the proposal model is symmetric.

Step 1: Propose a random location, u', for the next tour stop
Step 2: Decide whether to go to the proposed location ($u^{t-1}=u^{\prime }$ ) or to stay at the current location for another iteration ($u^{t+1}=u$)

We can propose Markov chain tour stops by sampling from a more different model such as a uniform proposal model. If the proposed location is greater than that of the current location- definitely go there. Otherwise, maybe go there?

Independence sampling algorithm

The same proposal model is utilized at each iteration, independent of the chain's current location.

Diagnostic

Trace plot

An example of a good trace-plot

Density plots of parallel chains

Effective sample ratio

Autocorrelation

See Autocorrelation

R-hat

Calculates the ratio between sources of variability - across all chains combined and then variability within any individual chain.

If R-hat = 1, this reflects stability. In contrast, R-hat > 1 indicates instability. R-hat ratio greater than 1.05 raises some red flags about the stability of the simulation

Program

Stan uses Hamiltonian Monte Carlo
Jags uses Gibb sampling

References

Bayes rules!