This is an old version, view current version.

## 20.3 Component Collapsing in Mixture Models

It is possible for two mixture components in a mixture model to collapse to the same values during sampling or optimization. For example, a mixture of $$K$$ normals might devolve to have $$\mu_i = \mu_j$$ and $$\sigma_i = \sigma_j$$ for $$i \neq j$$.

This will typically happen early in sampling due to initialization in MCMC or optimization or arise from random movement during MCMC. Once the parameters match for a given draw $$(m)$$, it can become hard to escape because there can be a trough of low-density mass between the current parameter values and the ones without collapsed components.

It may help to use a smaller step size during warmup, a stronger prior on each mixture component’s membership responsibility. A more extreme measure is to include additional mixture components to deal with the possibility that some of them may collapse.

In general, it is difficult to recover exactly the right $$K$$ mixture components in a mixture model as $$K$$ increases beyond one (yes, even a two-component mixture can have this problem).