7.3 Increment Log Density
The basis of Stan’s execution is the evaluation of a log probability function (specifically, a probability density function) for a given set of (real-valued) parameters; this function returns the log density of the posterior up to an additive constant. Data and transformed data are fixed before the log density is evaluated. The total log probability is initialized to zero. Next, any log Jacobian adjustments accrued by the variable constraints are added to the log density (the Jacobian adjustment may be skipped for optimization). Sampling and log probability increment statements may add to the log density in the model block. A log probability increment statement directly increments the log density with the value of an expression as follows.6
target += -0.5 * y * y;
The keyword target
here is actually not a variable, and may not
be accessed as such (though see below on how to access the value of
target through a special function).
In this example, the unnormalized log probability of a unit normal variable \(y\) is added to the total log probability. In the general case, the argument can be any expression.7
An entire Stan model can be implemented this way. For instance, the following model will draw a single variable according to a unit normal probability.
parameters {
real y;
}
model {
target += -0.5 * y * y;
}
This model defines a log probability function
\[ \log p(y) = - \, \frac{y^2}{2} - \log Z \]
where \(Z\) is a normalizing constant that does not depend on \(y\). The constant \(Z\) is conventionally written this way because on the linear scale, \[ p(y) = \frac{1}{Z} \exp\left(-\frac{y^2}{2}\right). \] which is typically written without reference to \(Z\) as \[ p(y) \propto \exp\left(-\frac{y^2}{2}\right). \]
Stan only requires models to be defined up to a constant that does not depend on the parameters. This is convenient because often the normalizing constant \(Z\) is either time-consuming to compute or intractable to evaluate.
Relation to compound addition and assignment
The increment log density statement looks syntactically like compound
addition and assignment (see the compound arithmetic/assignment section, it is treated as a
primitive statement because target
is not itself a variable.
So, even though
target += lp;
is a legal statement, the corresponding long form is not legal.
target = target + lp; // BAD, target is not a variable
Vectorization
The target += ...
statement accepts an argument in place of
...
for any expression type, including integers, reals,
vectors, row vectors, matrices, and arrays of any dimensionality,
including arrays of vectors and matrices. For container arguments,
their sum will be added to the total log density.
Accessing the Log Density
To access accumulated log density up to the current execution point,
the function target()()
may be used.
The current notation replaces two previous versions. Originally, a variable
lp__
was directly exposed and manipulated; this is no longer allowed. The original statement syntax fortarget += u
wasincrement_log_prob(u)
, but this form has been deprecated and will be removed in Stan 3.↩︎Writing this model with the expression
-0.5 * y * y
is more efficient than with the equivalent expressiony * y / -2
because multiplication is more efficient than division; in both cases, the negation is rolled into the numeric literal (-0.5
and-2
). Writingsquare(y)
instead ofy * y
would be even more efficient because the derivatives can be precomputed, reducing the memory and number of operations required for automatic differentiation.↩︎