13 Diagnosing HMC by Comparison of Gradients
CmdStan has a basic diagnostic feature that will calculate the gradients of the initial state and compare them with gradients calculated by finite differences. Discrepancies between the two indicate that there is a problem with the model or initial states or else there is a bug in Stan.
To allow for the possibility of adding other kinds of diagnostic tests, the diagnose
method
argument configuration has subargument test
which currently only takes value gradient
.
There are two available gradient test configuration arguments:
epsilon
- The finite difference step size. Must be a positive real number. Default value is \(1^{-6}\)error
- The error threshold. Must be a positive real number. Default value is \(1^{-6}\)
To run on the different platforms with the default configuration, use one of the following.
Mac OS and Linux
> ./my_model diagnose data file=my_data
Windows
> my_model diagnose data file=my_data
To relax the test threshold, specify the error
argument as follows:
> ./my_model diagnose test=gradient error=0.0001 data file=my_data
To see how this works, we run diagnostics on the example bernoulli model:
> ./bernoulli diagnose data file=bernoulli.data.R
Executing this command prints output to the console and as a series of comment lines to the output csv file. The console output is:
method = diagnose
diagnose
test = gradient (Default)
gradient
epsilon = 9.9999999999999995e-07 (Default)
error = 9.9999999999999995e-07 (Default)
id = 0 (Default)
data
file = bernoulli.data.json
init = 2 (Default)
random
seed = 2152196153 (Default)
output
file = output.csv (Default)
diagnostic_file = (Default)
refresh = 100 (Default)
TEST GRADIENT MODE
Log probability=-8.42814
param idx value model finite diff error
0 0.0361376 -3.1084 -3.1084 -2.37554e-10
The same information is printed to the output file as csv comments, i.e.,
each line is prefixed with a pound sign #
.