`stansummary`

: MCMC Output Analysis

The CmdStan `stansummary`

program reports statistics for one or more sampler chains over all sampler and model parameters and quantities of interest. The statistics reported include both summary statistics of the estimates and diagnostic statistics on the sampler chains, reported in the following order:

- Mean - sample mean
- MCSE - Monte Carlo Standard Error, a measure of the amount of noise in the sample
- StdDev - sample standard deviation
- Quantiles - default 5%, 50%, 95%
- N_eff - effective sample size - the number of independent draws in the sample
- N_eff/S - the number of independent draws per second
- R_hat - \(\hat{R}\) statistic, a measure of chain equilibrium, must be within \(0.05\) of \(1.0\).

When reviewing the `stansummary`

output, it is important to check the final three output columns first - these are the diagnostic statistics on chain convergence and number of independent draws in the sample. A \(\hat{R}\) statistic of greater than \(1.05\) indicates that the chain has not converged and therefore the sample is not drawn from the posterior, thus the estimates of the mean and all other summary statistics are invalid.

Estimation by sampling produces an approximate value for the model parameters; the MCSE statistic indicates the amount of noise in the estimate. Therefore MCSE column is placed next to the sample mean column, in order to make it easy to compare this sample with others.

For more information, see the Posterior Analysis chapter of the Stan Reference Manual which describes both the theory and practice of MCMC estimation techniques. The summary statistics and the algorithms used to compute them are described in sections Notation for samples and Effective Sample Size.

## Building the stansummary command

The CmdStan makefile task `build`

compiles the `stansummary`

utility into the `bin`

directory. It can be compiled directly using the makefile as follows:

```
> cd <cmdstan-home>
> make bin/stansummary
```

## Running the `stansummary`

program

The `stansummary`

utility processes one or more output files from a set of chains from one run of the HMC sampler. To run `stansummary`

on the output file or files generated by a run of the sampler, on Mac or Linux:

`<cmdstan-home>/bin/stansummary <file_1.csv> ... <file_N.csv>`

On Windows, use backslashes to call the `stansummary.exe`

.

`<cmdstan-home>\bin\stansummary.exe <file_1.csv> ... <file_N.csv>`

For example, after running 4 chains to fit the example model `eight_schools.stan`

to the supplied example data file, we run `stansummary`

on the resulting Stan CSV output files to get the following report:

```
> bin/stansummary eight_*.csv
Input files: eight_1.csv, eight_2.csv, eight_3.csv, eight_4.csv
Inference for Stan model: eight_schools_model
4 chains: each with iter=(1000,1000,1000,1000); warmup=(0,0,0,0); thin=(1,1,1,1); 4000 iterations saved.
Warmup took (0.048, 0.060, 0.047, 0.045) seconds, 0.20 seconds total
Sampling took (0.057, 0.058, 0.061, 0.066) seconds, 0.24 seconds total
Mean MCSE StdDev 5% 50% 95% N_Eff N_Eff/s R_hat
lp__ -18 0.33 5.1 -26 -19 -9.1 233 963 1.0
accept_stat__ 0.88 1.6e-02 0.23 0.21 0.98 1.00 203 838 1.0e+00
stepsize__ 0.18 2.2e-02 0.031 0.14 0.20 0.22 2.0 8.3 3.9e+13
treedepth__ 3.8 5.9e-02 0.78 2.0 4.0 5.0 175 724 1.0e+00
n_leapfrog__ 18 1.3e+00 9.4 7.0 15 31 51 212 1.0e+00
divergent__ 0.015 4.1e-03 0.12 0.00 0.00 0.00 865 3576 1.0e+00
energy__ 23 3.4e-01 5.5 13 23 32 258 1066 1.0e+00
mu 7.9 0.16 5.1 -0.23 7.9 16 1021 4221 1.0
theta[1] 12 0.30 8.6 -0.48 11 28 837 3459 1.0
theta[2] 7.8 0.15 6.4 -2.7 7.7 18 1717 7096 1.00
theta[3] 6.1 0.19 7.7 -6.5 6.5 18 1684 6958 1.0
theta[4] 7.5 0.15 6.7 -3.1 7.4 18 2026 8373 1.0
theta[5] 4.7 0.17 6.4 -6.7 5.3 15 1391 5747 1.00
theta[6] 5.9 0.16 6.7 -5.8 6.2 16 1673 6915 1.00
theta[7] 11 0.22 7.0 0.057 10 23 1069 4419 1.0
theta[8] 8.3 0.20 7.9 -4.2 8.0 22 1503 6209 1.00
tau 7.2 0.26 5.2 1.5 5.9 17 401 1657 1.0
Samples were drawn using hmc with nuts.
For each parameter, N_Eff is a crude measure of effective sample size,
and R_hat is the potential scale reduction factor on split chains (at
convergence, R_hat=1).
```

The console output information consists of

- Model, chains, and timing summaries
- Sampler parameter statistics
- Model parameter statistics
- Sampling algorithm - either nuts (shown here) or static HMC.

There is one row per parameter and the row order in the summary report corresponds to the column order in the Stan CSV output file.

### Sampler parameters

The initial Stan CSV columns provide information on the sampler state for each draw:

`lp__`

- the total log probability density (up to an additive constant) at each sample`accept_stat__`

- the average Metropolis acceptance probability over each simulated Hamiltonian trajectory`stepsize__`

- integrator step size`treedepth__`

- depth of tree used by NUTS (NUTS sampler)`n_leapfrog__`

- number of leapfrog calculations (NUTS sampler)`divergent__`

- has value`1`

if trajectory diverged, otherwise`0`

. (NUTS sampler)`energy__`

- value of the Hamiltonian`int_time__`

- total integration time (static HMC sampler)

Because we ran the NUTS sampler, the above summary reports sampler parameters `treedepth__`

, `n_leapfrog__`

, and `divergent__`

; the static HMC sampler would report `int_time__`

instead.

### Model parameters and quantities of interest

The remaining Stan CSV columns report the values of all parameters, transformed parameters, and generated quantities in the order in which these variables are declared in the Stan program. For container variables, i.e., vector, row_vector, matrix, and array variables, the statistics for each element are reported separately, in row-major order. The `eight_schools.stan`

program parameters block contains the following parameter variable declarations:

```
real mu;
array[J] real theta;
real<lower=0> tau;
```

In the example data, `J`

is \(8\); therefore the stansummary listing reports on `theta[1]`

through `theta[8]`

.

## Command-line options

The `stansummary`

command syntax provides a set of flags to customize the output which must precede the list of filenames. When invoked with no arguments or with the `-h`

or `--help`

option, the program prints the usage message to the console and exits.

```
Report statistics for one or more Stan CSV files from a HMC sampler run.
Example: stansummary model_chain_1.csv model_chain_2.csv
Options:
-a, --autocorr [n] Display the chain autocorrelation for the n-th
input file, in addition to statistics.
-c, --csv_filename [file] Write statistics to a CSV file.
-h, --help Produce help message, then exit.
-p, --percentiles [values] Percentiles to report as ordered set of
comma-separated numbers from (0.1,99.9), inclusive.
Default is 5,50,95.
-s, --sig_figs [n] Significant figures reported. Default is 2.
Must be an integer from (1, 18), inclusive.
-i, --include_param [name] Include the named parameter in the summary output.
By default, all parameters in the file are summarized,
passing this argument one or more times will filter
the output down to just the requested arguments.
```

Both short an long option names are allowed. Short names are specified as `-<o> <value>`

; long option names can be specified either as `--<option>=<value>`

or `--<option> <value>`

.

The `--percentiles`

argument can also be passed an empty string `""`

, which results in no percentiles being displayed in the output of the command.

The amount of precision in the sampler output limits the amount of real precision in the summary report. CmdStan’s command line interface also has output argument `sig_figs`

. The default sampler output precision is 6. The `--sig_figs`

argument to the stansummary program should not exceed the `sig_figs`

argument to the sampler.