Controlling Outputs¶
CSV File Outputs¶
Underlyingly, the CmdStan outputs are a set of per-chain Stan CSV files. The filenames follow the template ‘<model_name>-<YYYYMMDDHHMMSS>-<chain_id>’ plus the file suffix ‘.csv’. CmdStanPy also captures the per-chain console and error messages.
In [1]: import os
In [2]: from cmdstanpy import CmdStanModel
In [3]: stan_file = os.path.join('users-guide', 'examples', 'bernoulli.stan')
In [4]: model = CmdStanModel(stan_file=stan_file)
In [5]: data_file = os.path.join('users-guide', 'examples', 'bernoulli.data.json')
In [6]: fit = model.sample(data=data_file)
INFO:cmdstanpy:CmdStan start processing
INFO:cmdstanpy:CmdStan done processing.
# printing the object reports sampler commands, output files
In [7]: print(fit)
CmdStanMCMC: model=bernoulli chains=4['method=sample', 'algorithm=hmc', 'adapt', 'engaged=1']
csv_files:
/tmp/tmpz6k203wo/bernoulliojkfgzo6/bernoulli-20240617142916_1.csv
/tmp/tmpz6k203wo/bernoulliojkfgzo6/bernoulli-20240617142916_2.csv
/tmp/tmpz6k203wo/bernoulliojkfgzo6/bernoulli-20240617142916_3.csv
/tmp/tmpz6k203wo/bernoulliojkfgzo6/bernoulli-20240617142916_4.csv
output_files:
/tmp/tmpz6k203wo/bernoulliojkfgzo6/bernoulli-20240617142916_0-stdout.txt
/tmp/tmpz6k203wo/bernoulliojkfgzo6/bernoulli-20240617142916_1-stdout.txt
/tmp/tmpz6k203wo/bernoulliojkfgzo6/bernoulli-20240617142916_2-stdout.txt
/tmp/tmpz6k203wo/bernoulliojkfgzo6/bernoulli-20240617142916_3-stdout.txt
The output_dir argument is an optional argument which specifies
the path to the output directory used by CmdStan.
If this argument is omitted, the output files are written
to a temporary directory which is deleted when the current Python session is terminated.
In [8]: fit = model.sample(data=data_file, output_dir="./outputs/")
INFO:cmdstanpy:created output directory: /home/runner/work/cmdstanpy/cmdstanpy/docsrc/outputs
INFO:cmdstanpy:CmdStan start processing
INFO:cmdstanpy:CmdStan done processing.
In [9]: !ls outputs/
bernoulli-20240617142916_0-stdout.txt bernoulli-20240617142916_2.csv
bernoulli-20240617142916_1-stdout.txt bernoulli-20240617142916_3-stdout.txt
bernoulli-20240617142916_1.csv bernoulli-20240617142916_3.csv
bernoulli-20240617142916_2-stdout.txt bernoulli-20240617142916_4.csv
Alternatively, the save_csvfiles() function moves the CSV files
to a specified directory.
In [10]: fit = model.sample(data=data_file)
INFO:cmdstanpy:CmdStan start processing
INFO:cmdstanpy:CmdStan done processing.
In [11]: fit.save_csvfiles(dir='some/path')
In [12]: !ls some/path
bernoulli-20240617142917_1.csv bernoulli-20240617142917_3.csv
bernoulli-20240617142917_2.csv bernoulli-20240617142917_4.csv
Logging¶
You may notice CmdStanPy can produce a lot of output when it is running:
In [13]: fit = model.sample(data=data_file, show_progress=False)
INFO:cmdstanpy:CmdStan start processing
INFO:cmdstanpy:Chain [1] start processing
INFO:cmdstanpy:Chain [2] start processing
INFO:cmdstanpy:Chain [3] start processing
INFO:cmdstanpy:Chain [4] start processing
INFO:cmdstanpy:Chain [3] done processing
INFO:cmdstanpy:Chain [1] done processing
INFO:cmdstanpy:Chain [4] done processing
INFO:cmdstanpy:Chain [2] done processing
This output is managed through the built-in logging module. For example, it can be disabled entirely:
In [14]: import logging
In [15]: cmdstanpy_logger = logging.getLogger("cmdstanpy")
In [16]: cmdstanpy_logger.disabled = True
# look, no output!
In [17]: fit = model.sample(data=data_file, show_progress=False)
Or one can remove the logging handler that CmdStanPy installs by default and install their own for more
fine-grained control. For example, the following code sends all logs (including the DEBUG logs, which are hidden by default),
to a file.
DEBUG logging is useful primarily to developers or when trying to hunt down an issue.
In [18]: cmdstanpy_logger.disabled = False
# remove all existing handlers
In [19]: cmdstanpy_logger.handlers = []
In [20]: cmdstanpy_logger.setLevel(logging.DEBUG)
In [21]: handler = logging.FileHandler('all.log')
In [22]: handler.setLevel(logging.DEBUG)
In [23]: handler.setFormatter(
....: logging.Formatter(
....: '%(asctime)s - %(name)s - %(levelname)s - %(message)s',
....: "%H:%M:%S",
....: )
....: )
....:
In [24]: cmdstanpy_logger.addHandler(handler)
Now, if we run the model and check the contents of the file, we will see all the possible logging.
In [25]: fit = model.sample(data=data_file, show_progress=False)
In [26]: with open('all.log','r') as logs:
....: for line in logs.readlines():
....: print(line.strip())
....:
14:29:17 - cmdstanpy - DEBUG - cmd: /home/runner/work/cmdstanpy/cmdstanpy/docsrc/users-guide/examples/bernoulli info
cwd: None
14:29:17 - cmdstanpy - INFO - CmdStan start processing
14:29:17 - cmdstanpy - DEBUG - idx 0
14:29:17 - cmdstanpy - DEBUG - idx 1
14:29:17 - cmdstanpy - DEBUG - running CmdStan, num_threads: 1
14:29:17 - cmdstanpy - DEBUG - idx 2
14:29:17 - cmdstanpy - DEBUG - idx 3
14:29:17 - cmdstanpy - DEBUG - running CmdStan, num_threads: 1
14:29:17 - cmdstanpy - DEBUG - CmdStan args: ['/home/runner/work/cmdstanpy/cmdstanpy/docsrc/users-guide/examples/bernoulli', 'id=1', 'random', 'seed=11039', 'data', 'file=users-guide/examples/bernoulli.data.json', 'output', 'file=/tmp/tmpz6k203wo/bernoulliwu4xc0ot/bernoulli-20240617142917_1.csv', 'method=sample', 'algorithm=hmc', 'adapt', 'engaged=1']
14:29:17 - cmdstanpy - DEBUG - running CmdStan, num_threads: 1
14:29:17 - cmdstanpy - DEBUG - running CmdStan, num_threads: 1
14:29:17 - cmdstanpy - DEBUG - CmdStan args: ['/home/runner/work/cmdstanpy/cmdstanpy/docsrc/users-guide/examples/bernoulli', 'id=2', 'random', 'seed=11039', 'data', 'file=users-guide/examples/bernoulli.data.json', 'output', 'file=/tmp/tmpz6k203wo/bernoulliwu4xc0ot/bernoulli-20240617142917_2.csv', 'method=sample', 'algorithm=hmc', 'adapt', 'engaged=1']
14:29:17 - cmdstanpy - INFO - Chain [1] start processing
14:29:17 - cmdstanpy - DEBUG - CmdStan args: ['/home/runner/work/cmdstanpy/cmdstanpy/docsrc/users-guide/examples/bernoulli', 'id=3', 'random', 'seed=11039', 'data', 'file=users-guide/examples/bernoulli.data.json', 'output', 'file=/tmp/tmpz6k203wo/bernoulliwu4xc0ot/bernoulli-20240617142917_3.csv', 'method=sample', 'algorithm=hmc', 'adapt', 'engaged=1']
14:29:17 - cmdstanpy - DEBUG - CmdStan args: ['/home/runner/work/cmdstanpy/cmdstanpy/docsrc/users-guide/examples/bernoulli', 'id=4', 'random', 'seed=11039', 'data', 'file=users-guide/examples/bernoulli.data.json', 'output', 'file=/tmp/tmpz6k203wo/bernoulliwu4xc0ot/bernoulli-20240617142917_4.csv', 'method=sample', 'algorithm=hmc', 'adapt', 'engaged=1']
14:29:17 - cmdstanpy - INFO - Chain [2] start processing
14:29:17 - cmdstanpy - INFO - Chain [3] start processing
14:29:17 - cmdstanpy - INFO - Chain [4] start processing
14:29:17 - cmdstanpy - INFO - Chain [1] done processing
14:29:17 - cmdstanpy - INFO - Chain [4] done processing
14:29:17 - cmdstanpy - INFO - Chain [3] done processing
14:29:17 - cmdstanpy - INFO - Chain [2] done processing
14:29:17 - cmdstanpy - DEBUG - runset
RunSet: chains=4, chain_ids=[1, 2, 3, 4], num_processes=4
cmd (chain 1):
['/home/runner/work/cmdstanpy/cmdstanpy/docsrc/users-guide/examples/bernoulli', 'id=1', 'random', 'seed=11039', 'data', 'file=users-guide/examples/bernoulli.data.json', 'output', 'file=/tmp/tmpz6k203wo/bernoulliwu4xc0ot/bernoulli-20240617142917_1.csv', 'method=sample', 'algorithm=hmc', 'adapt', 'engaged=1']
retcodes=[0, 0, 0, 0]
per-chain output files (showing chain 1 only):
csv_file:
/tmp/tmpz6k203wo/bernoulliwu4xc0ot/bernoulli-20240617142917_1.csv
console_msgs (if any):
/tmp/tmpz6k203wo/bernoulliwu4xc0ot/bernoulli-20240617142917_0-stdout.txt
14:29:17 - cmdstanpy - DEBUG - Chain 1 console:
method = sample (Default)
sample
num_samples = 1000 (Default)
num_warmup = 1000 (Default)
save_warmup = false (Default)
thin = 1 (Default)
adapt
engaged = true (Default)
gamma = 0.05 (Default)
delta = 0.8 (Default)
kappa = 0.75 (Default)
t0 = 10 (Default)
init_buffer = 75 (Default)
term_buffer = 50 (Default)
window = 25 (Default)
save_metric = false (Default)
algorithm = hmc (Default)
hmc
engine = nuts (Default)
nuts
max_depth = 10 (Default)
metric = diag_e (Default)
metric_file = (Default)
stepsize = 1 (Default)
stepsize_jitter = 0 (Default)
num_chains = 1 (Default)
id = 1 (Default)
data
file = users-guide/examples/bernoulli.data.json
init = 2 (Default)
random
seed = 11039
output
file = /tmp/tmpz6k203wo/bernoulliwu4xc0ot/bernoulli-20240617142917_1.csv
diagnostic_file = (Default)
refresh = 100 (Default)
sig_figs = -1 (Default)
profile_file = profile.csv (Default)
save_cmdstan_config = false (Default)
num_threads = 1 (Default)
Gradient evaluation took 3e-06 seconds
1000 transitions using 10 leapfrog steps per transition would take 0.03 seconds.
Adjust your expectations accordingly!
Iteration: 1 / 2000 [ 0%] (Warmup)
Iteration: 100 / 2000 [ 5%] (Warmup)
Iteration: 200 / 2000 [ 10%] (Warmup)
Iteration: 300 / 2000 [ 15%] (Warmup)
Iteration: 400 / 2000 [ 20%] (Warmup)
Iteration: 500 / 2000 [ 25%] (Warmup)
Iteration: 600 / 2000 [ 30%] (Warmup)
Iteration: 700 / 2000 [ 35%] (Warmup)
Iteration: 800 / 2000 [ 40%] (Warmup)
Iteration: 900 / 2000 [ 45%] (Warmup)
Iteration: 1000 / 2000 [ 50%] (Warmup)
Iteration: 1001 / 2000 [ 50%] (Sampling)
Iteration: 1100 / 2000 [ 55%] (Sampling)
Iteration: 1200 / 2000 [ 60%] (Sampling)
Iteration: 1300 / 2000 [ 65%] (Sampling)
Iteration: 1400 / 2000 [ 70%] (Sampling)
Iteration: 1500 / 2000 [ 75%] (Sampling)
Iteration: 1600 / 2000 [ 80%] (Sampling)
Iteration: 1700 / 2000 [ 85%] (Sampling)
Iteration: 1800 / 2000 [ 90%] (Sampling)
Iteration: 1900 / 2000 [ 95%] (Sampling)
Iteration: 2000 / 2000 [100%] (Sampling)
Elapsed Time: 0.005 seconds (Warm-up)
0.014 seconds (Sampling)
0.019 seconds (Total)