This is an old version, view current version.

# 26 Hidden Markov Models

An elementary first-order Hidden Markov model is a probabilistic model over $$N$$ observations, $$y_n$$, and $$N$$ hidden states, $$x_n$$, which can be fully defined by the conditional distributions $$p(y_n \mid x_n, \phi)$$ and $$p(x_n \mid x_{n - 1}, \phi)$$. Here we make the dependency on additional model parameters, $$\phi$$, explicit. When $$x$$ is continuous, the user can explicitly encode these distributions in Stan and use Markov chain Monte Carlo to integrate $$x$$ out.

When each state $$x$$ takes a value over a discrete and finite set, say $$\{1, 2, ..., K\}$$, we can take advantage of the dependency structure to marginalize $$x$$ and compute $$p(y \mid \phi)$$. We start by defining the conditional observational distribution, stored in a $$K \times N$$ matrix $$\omega$$ with $\omega_{kn} = p(y_n \mid x_n = k, \phi).$ Next, we introduce the $$K \times K$$ transition matrix, $$\Gamma$$, with $\Gamma_{ij} = p(x_n = j \mid x_{n - 1} = i, \phi).$ Each row defines a probability distribution and must therefore be a simplex (i.e. its components must add to 1). Currently, Stan only supports stationary transitions where a single transition matrix is used for all transitions. Finally we define the initial state $$K$$-vector $$\rho$$, with $\rho_k = p(x_0 = k \mid \phi).$

The Stan functions that support this type of model are special in that the user does not explicitly pass $$y$$ and $$\phi$$ as arguments. Instead, the user passes $$\log \omega$$, $$\Gamma$$, and $$\rho$$, which in turn depend on $$y$$ and $$\phi$$.