6.12 Special matrix functions
6.12.1 Softmax
The softmax function maps3 y∈RK to the K-simplex by softmax(y)=exp(y)∑Kk=1exp(yk), where exp(y) is the componentwise exponentiation of y. Softmax is usually calculated on the log scale, logsoftmax(y)= y−logK∑k=1exp(yk)=y−log_sum_exp(y). where the vector y minus the scalar log_sum_exp(y) subtracts the scalar from each component of y.
Stan provides the following functions for softmax and its log.
vector
softmax
(vector x)
The softmax of x
Available since 2.0
vector
log_softmax
(vector x)
The natural logarithm of the softmax of x
Available since 2.0
6.12.2 Cumulative sums
The cumulative sum of a sequence x1,…,xN is the sequence y1,…,yN, where yn=n∑m=1xm.
array[] int
cumulative_sum
(array[] int x)
The cumulative sum of x
Available since 2.30
array[] real
cumulative_sum
(array[] real x)
The cumulative sum of x
Available since 2.0
vector
cumulative_sum
(vector v)
The cumulative sum of v
Available since 2.0
row_vector
cumulative_sum
(row_vector rv)
The cumulative sum of rv
Available since 2.0
The softmax function is so called because in the limit as yn→∞ with ym for m≠n held constant, the result tends toward the “one-hot” vector θ with θn=1 and θm=0 for m≠n, thus providing a “soft” version of the maximum function.↩︎