Diagnosing HMC by Comparison of Gradients

CmdStan has a basic diagnostic feature that will calculate the gradients of the initial state and compare them with gradients calculated by finite differences. Discrepancies between the two indicate that there is a problem with the model or initial states or else there is a bug in Stan.

To allow for the possibility of adding other kinds of diagnostic tests, the diagnose method argument configuration has subargument test which currently only takes value gradient. There are two available gradient test configuration arguments:

  • epsilon - The finite difference step size. Must be a positive real number. Default value is \(1^{-6}\)

  • error - The error threshold. Must be a positive real number. Default value is \(1^{-6}\)

To run on the different platforms with the default configuration, use one of the following.

Mac OS and Linux

> ./my_model diagnose data file=my_data

Windows

> my_model diagnose data file=my_data

To relax the test threshold, specify the error argument as follows:

> ./my_model diagnose test=gradient error=0.0001 data file=my_data

To see how this works, we run diagnostics on the example bernoulli model:

> ./bernoulli diagnose data file=bernoulli.data.R

Executing this command prints output to the console and as a series of comment lines to the output csv file. The console output is:

method = diagnose
  diagnose
    test = gradient (Default)
      gradient
        epsilon = 9.9999999999999995e-07 (Default)
        error = 9.9999999999999995e-07 (Default)
id = 0 (Default)
data
  file = bernoulli.data.json
init = 2 (Default)
random
  seed = 2152196153 (Default)
output
  file = output.csv (Default)
  diagnostic_file =  (Default)
  refresh = 100 (Default)

TEST GRADIENT MODE

 Log probability=-8.42814

 param idx           value           model     finite diff           error
         0       0.0361376         -3.1084         -3.1084    -2.37554e-10

The same information is printed to the output file as csv comments, i.e., each line is prefixed with a pound sign #.

Back to top