This is an old version, view current version.
5.12 Special matrix functions
5.12.1 Softmax
The softmax function maps3 y∈RK to the K-simplex by softmax(y)=exp(y)∑Kk=1exp(yk), where exp(y) is the componentwise exponentiation of y. Softmax is usually calculated on the log scale, logsoftmax(y)= y−logK∑k=1exp(yk)=y−log_sum_exp(y). where the vector y minus the scalar log_sum_exp(y) subtracts the scalar from each component of y.
Stan provides the following functions for softmax and its log.
vector
softmax
(vector x)
The softmax of x
vector
log_softmax
(vector x)
The natural logarithm of the softmax of x
The softmax function is so called because in the limit as yn→∞ with ym for m≠n held constant, the result tends toward the “one-hot” vector θ with θn=1 and θm=0 for m≠n, thus providing a “soft” version of the maximum function.↩︎