12.2 Initialization
Initialization is the same for sampling, optimization, and diagnosis
User-supplied initial values
If there are user-supplied initial values for parameters, these are read using the same input mechanism and same file format as data reads. Any constraints declared on the parameters are validated for the initial values. If a variable’s value violates its declared constraint, the program halts and a diagnostic message is printed.
After being read, initial values are transformed to unconstrained values that will be used to initialize the sampler.
Boundary values are problematic
Because of the way Stan defines its transforms from the constrained to the unconstrained space, initializing parameters on the boundaries of their constraints is usually problematic. For instance, with a constraint
parameters {
real<lower=0, upper=1> theta;
// ...
}an initial value of 0 for theta leads to an unconstrained value
of \(-\infty\), whereas a value of 1 leads to an unconstrained value of
\(+\infty\). While this will be inverse transformed back correctly
given the behavior of floating point arithmetic, the Jacobian will be
infinite and the log probability function will fail and raise an
exception.
Random initial values
If there are no user-supplied initial values, the default initialization strategy is to initialize the unconstrained parameters directly with values drawn uniformly from the interval \((-2,2)\). The bounds of this initialization can be changed but it is always symmetric around 0. The value of 0 is special in that it represents the median of the initialization. An unconstrained value of 0 corresponds to different parameter values depending on the constraints declared on the parameters.
An unconstrained real does not involve any transform, so an initial value of 0 for the unconstrained parameters is also a value of 0 for the constrained parameters.
For parameters that are bounded below at 0, the initial value of 0 on the unconstrained scale corresponds to \(\exp(0) = 1\) on the constrained scale. A value of -2 corresponds to \(\exp(-2) = .13\) and a value of 2 corresponds to \(\exp(2) = 7.4\).
For parameters bounded above and below, the initial value of 0 on the unconstrained scale corresponds to a value at the midpoint of the constraint interval. For probability parameters, bounded below by 0 and above by 1, the transform is the inverse logit, so that an initial unconstrained value of 0 corresponds to a constrained value of 0.5, -2 corresponds to 0.12 and 2 to 0.88. Bounds other than 0 and 1 are just scaled and translated.
Simplexes with initial values of 0 on the unconstrained basis correspond to symmetric values on the constrained values (i.e., each value is \(1/K\) in a \(K\)-simplex).
Cholesky factors for positive-definite matrices are initialized to 1 on the diagonal and 0 elsewhere; this is because the diagonal is log transformed and the below-diagonal values are unconstrained.
The initial values for other parameters can be determined from the transform that is applied. The transforms are all described in full detail in the chapter on variable transforms.
Zero initial values
The initial values may all be set to 0 on the unconstrained scale. This can be helpful for diagnosis, and may also be a good starting point for sampling. Once a model is running, multiple chains with more diffuse starting points can help diagnose problems with convergence; see the user’s guide for more information on convergence monitoring.