Conjugate Prior

Gist

If you choose a certain probability distribution for a prior and a likelihood. Your posterior distribution will have the same probability distribution of the prior. Useful for Bayesian analysis because you can then find an analytical solution.

Beta - Binomial

A Beta Distribution prior with a binomial likelihood leads go a beta posterior distribution.

Derivation

Gamma - Poisson

Normal-Normal

Dirichlet - Categorial distribution

Dirichlet - Multinomial distribution

Drawbacks

1. A conjugate prior isn't always flexible enough to fit your prior understanding!