This is an old version, view current version.

26.3 Sampling from the posterior predictive distribution

Given draws from the posterior $\theta^{(m)} \sim p(\theta \mid y),$ draws from the posterior predictive $\tilde{y}^{(m)} \sim p(\tilde{y} \mid y)$ can be generated by randomly generating from the sampling distribution with the parameter draw plugged in, $$\tilde{y}^{(m)} \sim p(y \mid \theta^{(m)}).$$

Randomly drawing $\tilde{y}$ from the sampling distribution is critical because there are two forms of uncertainty in posterior predictive quantities, sampling uncertainty and estimation uncertainty. Estimation uncertainty arises because $\theta$ is being estimated based only on a sample of data $y$. Sampling uncertainty arises because even a known value of $\theta$ leads to a sampling distribution $p(\tilde{y} \mid \theta)$ with variation in $\tilde{y}$. Both forms of uncertainty show up in the factored form of the posterior predictive distribution, $$p(\tilde{y} \mid y) = \int \underbrace{p(\tilde{y} \mid \theta)}_{\begin{array}{l} \textrm{sampling} \\[-2pt] \textrm{uncertainty} \end{array}} \cdot \underbrace{p(\theta \mid y)}_{\begin{array}{l} \textrm{estimation} \\[-2pt] \textrm{uncertainty} \end{array}} \, \textrm{d}\theta.$$