vignettes/cmdstanr-internals.Rmd
cmdstanr-internals.RmdThis vignette is intended to be read after the Getting started with CmdStanR vignette. Please read that first for important background. In this document we provide additional details about compiling models, passing in data, and how CmdStan output is saved and read back into R.
We will only use the $sample() method in examples, but
all model fitting methods work in a similar way under the hood.
library(cmdstanr)
check_cmdstan_toolchain(fix = TRUE, quiet = TRUE)The cmdstan_model() function creates a new
CmdStanModel object. The CmdStanModel object
stores the path to a Stan program as well as the path to a compiled
executable.
stan_file <- file.path(cmdstan_path(), "examples", "bernoulli", "bernoulli.stan")
mod <- cmdstan_model(stan_file)
mod$print()data {
int<lower=0> N;
array[N] int<lower=0,upper=1> y;
}
parameters {
real<lower=0,upper=1> theta;
}
model {
theta ~ beta(1,1); // uniform prior on interval 0,1
y ~ bernoulli(theta);
}
mod$stan_file()[1] "/Users/jgabry/.cmdstan/cmdstan-2.33.1/examples/bernoulli/bernoulli.stan"
mod$exe_file()[1] "/Users/jgabry/.cmdstan/cmdstan-2.33.1/examples/bernoulli/bernoulli"Subsequently, if you create a CmdStanModel object from
the same Stan file then compilation will be skipped (assuming the file
hasn’t changed).
mod <- cmdstan_model(stan_file)Internally, cmdstan_model() first creates the
CmdStanModel object from just the Stan file and then calls
its $compile()
method. Optional arguments to the $compile() method can be
passed via ....
mod <- cmdstan_model(
stan_file,
force_recompile = TRUE,
include_paths = "paths/to/directories/with/included/files",
cpp_options = list(stan_threads = TRUE, STANC2 = TRUE)
)It is also possible to delay compilation when creating the
CmdStanModel object by specifying
compile=FALSE and then later calling the
$compile() method directly.
unlink(mod$exe_file())
mod <- cmdstan_model(stan_file, compile = FALSE)
mod$exe_file() # not yet createdcharacter(0)
mod$compile()
mod$exe_file()[1] "/Users/jgabry/.cmdstan/cmdstan-2.33.1/examples/bernoulli/bernoulli"If you are using CmdStan version 2.24 or later and CmdStanR version 0.2.1 or later, you can run a pedantic check for your model. CmdStanR will always check that your Stan program does not contain any invalid syntax but with pedantic mode enabled the check will also warn you about other potential issues in your model, for example:
~ symbols).For the latest information on the checks performed in pedantic mode see the Pedantic mode chapter in the Stan Reference Manual.
Pedantic mode is available when compiling the model or when using the
separate $check_syntax() method of a
CmdStanModel object. Internally this corresponds to setting
the stanc (Stan transpiler) option
warn-pedantic. Here we demonstrate pedantic mode with a
Stan program that is syntactically correct but is missing a lower bound
and a prior for a parameter.
stan_file_pedantic <- write_stan_file("
data {
int N;
array[N] int y;
}
parameters {
// should have <lower=0> but omitting to demonstrate pedantic mode
real lambda;
}
model {
y ~ poisson(lambda);
}
")To turn on pedantic mode at compile time you can set
pedantic=TRUE in the call to cmdstan_model()
(or when calling the $compile() method directly if using
the delayed compilation approach described above).
mod_pedantic <- cmdstan_model(stan_file_pedantic, pedantic = TRUE)
Warning in '/var/folders/s0/zfzm55px2nd2v__zlw5xfj2h0000gn/T/RtmpXOnqh8/model-74281eb75594.stan', line 11, column 14: A
poisson distribution is given parameter lambda as a rate parameter
(argument 1), but lambda was not constrained to be strictly positive.
Warning: The parameter lambda has no priors. This means either no prior is
provided, or the prior(s) depend on data variables. In the later case,
this may be a false positive.To turn on pedantic mode separately from compilation use the
pedantic argument to the $check_syntax()
method.
mod_pedantic$check_syntax(pedantic = TRUE)
Warning in '/var/folders/s0/zfzm55px2nd2v__zlw5xfj2h0000gn/T/RtmpXOnqh8/model_febb1e69c7387a0e64cf13583e078104.stan', line 11, column 14: A
poisson distribution is given parameter lambda as a rate parameter
(argument 1), but lambda was not constrained to be strictly positive.
Warning: The parameter lambda has no priors. This means either no prior is
provided, or the prior(s) depend on data variables. In the later case,
this may be a false positive.
Stan program is syntactically correctUsing pedantic=TRUE via the $check_syntax()
method also has the advantage that it can be used even if the model
hasn’t been compiled yet. This can be helpful because the pedantic and
syntax checks themselves are much faster than compilation.
file.remove(mod_pedantic$exe_file()) # delete compiled executable
[1] TRUE
rm(mod_pedantic)
mod_pedantic <- cmdstan_model(stan_file_pedantic, compile = FALSE)
mod_pedantic$check_syntax(pedantic = TRUE)
Warning in '/var/folders/s0/zfzm55px2nd2v__zlw5xfj2h0000gn/T/RtmpXOnqh8/model_febb1e69c7387a0e64cf13583e078104.stan', line 11, column 14: A
poisson distribution is given parameter lambda as a rate parameter
(argument 1), but lambda was not constrained to be strictly positive.
Warning: The parameter lambda has no priors. This means either no prior is
provided, or the prior(s) depend on data variables. In the later case,
this may be a false positive.
Stan program is syntactically correctIf using CmdStan 2.27 or newer, you can obtain the names, types and
dimensions of the data, parameters, transformed parameters and generated
quantities variables of a Stan model using the $variables()
method of the CmdStanModel object.
stan_file_variables <- write_stan_file("
data {
int<lower=1> J;
vector<lower=0>[J] sigma;
vector[J] y;
}
parameters {
real mu;
real<lower=0> tau;
vector[J] theta_raw;
}
transformed parameters {
vector[J] theta = mu + tau * theta_raw;
}
model {
target += normal_lpdf(tau | 0, 10);
target += normal_lpdf(mu | 0, 10);
target += normal_lpdf(theta_raw | 0, 1);
target += normal_lpdf(y | theta, sigma);
}
")
mod_v <- cmdstan_model(stan_file_variables)
variables <- mod_v$variables()The $variables() method returns a list with
data, parameters,
transformed_parameters and
generated_quantities elements, each corresponding to
variables in their respective block of the program. Transformed data
variables are not listed as they are not used in the model’s input or
output.
names(variables)[1] "parameters" "included_files" "data"
[4] "transformed_parameters" "generated_quantities"
names(variables$data)[1] "J" "sigma" "y"
names(variables$parameters)[1] "mu" "tau" "theta_raw"
names(variables$transformed_parameters)[1] "theta"
names(variables$generated_quantities)character(0)Each variable is represented as a list containing the type
information (currently limited to real or int)
and the number of dimensions.
variables$data$J$type
[1] "int"
$dimensions
[1] 0
variables$data$sigma$type
[1] "real"
$dimensions
[1] 1
variables$parameters$tau$type
[1] "real"
$dimensions
[1] 0
variables$transformed_parameters$theta$type
[1] "real"
$dimensions
[1] 1By default, the executable is created in the same directory as the
file containing the Stan program. You can also specify a different
location with the dir argument.
mod <- cmdstan_model(stan_file, dir = "path/to/directory/for/executable")There are three data formats that CmdStanR allows when fitting a model:
Like the RStan interface, CmdStanR accepts a named list of R objects
where the names correspond to variables declared in the data block of
the Stan program. In the Bernoulli model the data is N, the
number of data points, and y an integer array of
observations.
mod$print()data {
int<lower=0> N;
array[N] int<lower=0,upper=1> y;
}
parameters {
real<lower=0,upper=1> theta;
}
model {
theta ~ beta(1,1); // uniform prior on interval 0,1
y ~ bernoulli(theta);
}
# data block has 'N' and 'y'
data_list <- list(N = 10, y = c(0,1,0,0,0,0,0,0,0,1))
fit <- mod$sample(data = data_list)Because CmdStan doesn’t accept lists of R objects, CmdStanR will
first write the data to a temporary JSON file using
write_stan_json(). This happens internally, but it is also
possible to call write_stan_json() directly.
data_list <- list(N = 10, y = c(0,1,0,0,0,0,0,0,0,1))
json_file <- tempfile(fileext = ".json")
write_stan_json(data_list, json_file)
cat(readLines(json_file), sep = "\n"){
"N": 10,
"y": [0, 1, 0, 0, 0, 0, 0, 0, 0, 1]
}If you already have your data in a JSON file you can just pass that
file directly to CmdStanR instead of using a list of R objects. For
example, we could pass in the JSON file we created above using
write_stan_json():
fit <- mod$sample(data = json_file)Finally, it is also possible to use the R dump file format. This is
not recommended because CmdStan can process JSON faster than R
dump, but CmdStanR allows it because CmdStan will accept files created
by rstan::stan_rdump():
When fitting a model, the default behavior is to write the output from CmdStan to CSV files in a temporary directory.
fit$output_files()[1] "/var/folders/s0/zfzm55px2nd2v__zlw5xfj2h0000gn/T/RtmpXOnqh8/bernoulli-202312131009-1-37e810.csv"
[2] "/var/folders/s0/zfzm55px2nd2v__zlw5xfj2h0000gn/T/RtmpXOnqh8/bernoulli-202312131009-2-37e810.csv"
[3] "/var/folders/s0/zfzm55px2nd2v__zlw5xfj2h0000gn/T/RtmpXOnqh8/bernoulli-202312131009-3-37e810.csv"
[4] "/var/folders/s0/zfzm55px2nd2v__zlw5xfj2h0000gn/T/RtmpXOnqh8/bernoulli-202312131009-4-37e810.csv"These files will be lost if you end your R session or if you remove
the fit object and force (or wait for) garbage
collection.
files <- fit$output_files()
file.exists(files)[1] TRUE TRUE TRUE TRUE used (Mb) gc trigger (Mb) limit (Mb) max used (Mb)
Ncells 1260600 67.4 2436292 130.2 NA 2436292 130.2
Vcells 2212085 16.9 8388608 64.0 32768 3459642 26.4
file.exists(files)[1] FALSE FALSE FALSE FALSETo save these files to a non-temporary location there are two
options. You can either specify the output_dir argument to
mod$sample() or use fit$save_output_files()
after fitting the model.
# see ?save_output_files for info on optional arguments
fit$save_output_files(dir = "path/to/directory")
fit <- mod$sample(
data = data_list,
output_dir = "path/to/directory"
)With the exception of some diagnostic information, the CSV files are
not read into R until their contents are requested by calling a method
that requires them (e.g., fit$draws(),
fit$summary(), etc.). If we examine the structure of the
fit object, notice how the Private slot
draws_ is NULL, indicating that the CSV files
haven’t yet been read into R.
str(fit)Classes 'CmdStanMCMC', 'CmdStanFit', 'R6' <CmdStanMCMC>
Inherits from: <CmdStanFit>
Public:
clone: function (deep = FALSE)
cmdstan_diagnose: function ()
cmdstan_summary: function (flags = NULL)
code: function ()
constrain_variables: function (unconstrained_variables, transformed_parameters = TRUE,
data_file: function ()
diagnostic_summary: function (diagnostics = c("divergences", "treedepth", "ebfmi"),
draws: function (variables = NULL, inc_warmup = FALSE, format = getOption("cmdstanr_draws_format",
expose_functions: function (global = FALSE, verbose = FALSE)
functions: environment
grad_log_prob: function (unconstrained_variables, jacobian = TRUE, jacobian_adjustment = NULL)
hessian: function (unconstrained_variables, jacobian = TRUE, jacobian_adjustment = TRUE)
init: function ()
init_model_methods: function (seed = 0, verbose = FALSE, hessian = FALSE)
initialize: function (runset)
inv_metric: function (matrix = TRUE)
latent_dynamics_files: function (include_failed = FALSE)
log_prob: function (unconstrained_variables, jacobian = TRUE, jacobian_adjustment = NULL)
loo: function (variables = "log_lik", r_eff = TRUE, moment_match = FALSE,
lp: function ()
metadata: function ()
num_chains: function ()
num_procs: function ()
output: function (id = NULL)
output_files: function (include_failed = FALSE)
print: function (variables = NULL, ..., digits = 2, max_rows = getOption("cmdstanr_max_rows",
profile_files: function (include_failed = FALSE)
profiles: function ()
return_codes: function ()
runset: CmdStanRun, R6
sampler_diagnostics: function (inc_warmup = FALSE, format = getOption("cmdstanr_draws_format",
save_data_file: function (dir = ".", basename = NULL, timestamp = TRUE, random = TRUE)
save_latent_dynamics_files: function (dir = ".", basename = NULL, timestamp = TRUE, random = TRUE)
save_object: function (file, ...)
save_output_files: function (dir = ".", basename = NULL, timestamp = TRUE, random = TRUE)
save_profile_files: function (dir = ".", basename = NULL, timestamp = TRUE, random = TRUE)
summary: function (variables = NULL, ...)
time: function ()
unconstrain_draws: function (files = NULL, draws = NULL)
unconstrain_variables: function (variables)
variable_skeleton: function (transformed_parameters = TRUE, generated_quantities = TRUE)
Private:
draws_: NULL
init_: NULL
inv_metric_: list
metadata_: list
model_methods_env_: environment
profiles_: NULL
read_csv_: function (variables = NULL, sampler_diagnostics = NULL, format = getOption("cmdstanr_draws_format",
return_codes_: 0 0 0 0
sampler_diagnostics_: 1 2 1 2 2 2 1 1 2 2 2 1 1 2 1 2 2 2 1 2 2 2 2 1 2 2 1 1 ...
warmup_draws_: NULL
warmup_sampler_diagnostics_: NULL After we call a method that requires the draws then if we reexamine
the structure of the object we will see that the draws_
slot in Private is no longer empty.
draws <- fit$draws() # force CSVs to be read into R
str(fit)Classes 'CmdStanMCMC', 'CmdStanFit', 'R6' <CmdStanMCMC>
Inherits from: <CmdStanFit>
Public:
clone: function (deep = FALSE)
cmdstan_diagnose: function ()
cmdstan_summary: function (flags = NULL)
code: function ()
constrain_variables: function (unconstrained_variables, transformed_parameters = TRUE,
data_file: function ()
diagnostic_summary: function (diagnostics = c("divergences", "treedepth", "ebfmi"),
draws: function (variables = NULL, inc_warmup = FALSE, format = getOption("cmdstanr_draws_format",
expose_functions: function (global = FALSE, verbose = FALSE)
functions: environment
grad_log_prob: function (unconstrained_variables, jacobian = TRUE, jacobian_adjustment = NULL)
hessian: function (unconstrained_variables, jacobian = TRUE, jacobian_adjustment = TRUE)
init: function ()
init_model_methods: function (seed = 0, verbose = FALSE, hessian = FALSE)
initialize: function (runset)
inv_metric: function (matrix = TRUE)
latent_dynamics_files: function (include_failed = FALSE)
log_prob: function (unconstrained_variables, jacobian = TRUE, jacobian_adjustment = NULL)
loo: function (variables = "log_lik", r_eff = TRUE, moment_match = FALSE,
lp: function ()
metadata: function ()
num_chains: function ()
num_procs: function ()
output: function (id = NULL)
output_files: function (include_failed = FALSE)
print: function (variables = NULL, ..., digits = 2, max_rows = getOption("cmdstanr_max_rows",
profile_files: function (include_failed = FALSE)
profiles: function ()
return_codes: function ()
runset: CmdStanRun, R6
sampler_diagnostics: function (inc_warmup = FALSE, format = getOption("cmdstanr_draws_format",
save_data_file: function (dir = ".", basename = NULL, timestamp = TRUE, random = TRUE)
save_latent_dynamics_files: function (dir = ".", basename = NULL, timestamp = TRUE, random = TRUE)
save_object: function (file, ...)
save_output_files: function (dir = ".", basename = NULL, timestamp = TRUE, random = TRUE)
save_profile_files: function (dir = ".", basename = NULL, timestamp = TRUE, random = TRUE)
summary: function (variables = NULL, ...)
time: function ()
unconstrain_draws: function (files = NULL, draws = NULL)
unconstrain_variables: function (variables)
variable_skeleton: function (transformed_parameters = TRUE, generated_quantities = TRUE)
Private:
draws_: -6.95442 -6.92494 -6.74858 -7.7961 -6.76129 -6.95584 -6. ...
init_: NULL
inv_metric_: list
metadata_: list
model_methods_env_: environment
profiles_: NULL
read_csv_: function (variables = NULL, sampler_diagnostics = NULL, format = getOption("cmdstanr_draws_format",
return_codes_: 0 0 0 0
sampler_diagnostics_: 1 2 1 2 2 2 1 1 2 2 2 1 1 2 1 2 2 2 1 2 2 2 2 1 2 2 1 1 ...
warmup_draws_: NULL
warmup_sampler_diagnostics_: NULL For models with many parameters, transformed parameters, or generated
quantities, if only some are requested (e.g., by specifying the
variables argument to fit$draws()) then
CmdStanR will only read in the requested variables (unless they have
already been read in).
Internally, the read_cmdstan_csv() function is used to
read the CmdStan CSV files into R. This function is exposed to users, so
you can also call it directly.
# see ?read_cmdstan_csv for info on optional arguments controlling
# what information is read in
csv_contents <- read_cmdstan_csv(fit$output_files())
str(csv_contents)List of 8
$ metadata :List of 40
..$ stan_version_major : num 2
..$ stan_version_minor : num 33
..$ stan_version_patch : num 0
..$ start_datetime : chr "2023-12-13 17:09:13 UTC"
..$ method : chr "sample"
..$ save_warmup : num 0
..$ thin : num 1
..$ gamma : num 0.05
..$ kappa : num 0.75
..$ t0 : num 10
..$ init_buffer : num 75
..$ term_buffer : num 50
..$ window : num 25
..$ algorithm : chr "hmc"
..$ engine : chr "nuts"
..$ metric : chr "diag_e"
..$ stepsize_jitter : num 0
..$ num_chains : num 1
..$ id : num [1:4] 1 2 3 4
..$ init : num [1:4] 2 2 2 2
..$ seed : num 1.03e+09
..$ refresh : num 100
..$ sig_figs : num -1
..$ profile_file : chr "/var/folders/s0/zfzm55px2nd2v__zlw5xfj2h0000gn/T/RtmpXOnqh8/bernoulli-profile-202312131009-1-714e4e.csv"
..$ stanc_version : chr "stanc3 v2.33.1"
..$ sampler_diagnostics : chr [1:6] "accept_stat__" "stepsize__" "treedepth__" "n_leapfrog__" ...
..$ variables : chr [1:2] "lp__" "theta"
..$ step_size_adaptation: num [1:4] 0.843 0.932 0.873 0.987
..$ model_name : chr "bernoulli_model"
..$ adapt_engaged : num 1
..$ adapt_delta : num 0.8
..$ max_treedepth : num 10
..$ step_size : num [1:4] 1 1 1 1
..$ iter_warmup : num 1000
..$ iter_sampling : num 1000
..$ threads_per_chain : num 1
..$ time :'data.frame': 4 obs. of 4 variables:
.. ..$ chain_id: num [1:4] 1 2 3 4
.. ..$ warmup : num [1:4] 0.004 0.005 0.005 0.004
.. ..$ sampling: num [1:4] 0.012 0.012 0.012 0.011
.. ..$ total : num [1:4] 0.016 0.017 0.017 0.015
..$ stan_variable_sizes :List of 2
.. ..$ lp__ : num 1
.. ..$ theta: num 1
..$ stan_variables : chr [1:2] "lp__" "theta"
..$ model_params : chr [1:2] "lp__" "theta"
$ time :List of 2
..$ total : int NA
..$ chains:'data.frame': 4 obs. of 4 variables:
.. ..$ chain_id: num [1:4] 1 2 3 4
.. ..$ warmup : num [1:4] 0.004 0.005 0.005 0.004
.. ..$ sampling: num [1:4] 0.012 0.012 0.012 0.011
.. ..$ total : num [1:4] 0.016 0.017 0.017 0.015
$ inv_metric :List of 4
..$ 1: num 0.497
..$ 2: num 0.514
..$ 3: num 0.571
..$ 4: num 0.489
$ step_size :List of 4
..$ 1: num 0.843
..$ 2: num 0.932
..$ 3: num 0.873
..$ 4: num 0.987
$ warmup_draws : NULL
$ post_warmup_draws : 'draws_array' num [1:1000, 1:4, 1:2] -6.95 -6.92 -6.75 -7.8 -6.76 ...
..- attr(*, "dimnames")=List of 3
.. ..$ iteration: chr [1:1000] "1" "2" "3" "4" ...
.. ..$ chain : chr [1:4] "1" "2" "3" "4"
.. ..$ variable : chr [1:2] "lp__" "theta"
$ warmup_sampler_diagnostics : NULL
$ post_warmup_sampler_diagnostics: 'draws_array' num [1:1000, 1:4, 1:6] 0.913 0.99 0.995 0.866 1 ...
..- attr(*, "dimnames")=List of 3
.. ..$ iteration: chr [1:1000] "1" "2" "3" "4" ...
.. ..$ chain : chr [1:4] "1" "2" "3" "4"
.. ..$ variable : chr [1:6] "accept_stat__" "stepsize__" "treedepth__" "n_leapfrog__" ...If you need to manually create fitted model objects from CmdStan CSV
files use as_cmdstan_fit().
fit2 <- as_cmdstan_fit(fit$output_files())This is pointless in our case since we have the original
fit object, but this function can be used to create fitted
model objects (CmdStanMCMC, CmdStanMLE, etc.)
from any CmdStan CSV files.
If save_latent_dynamics is set to TRUE when
running the $sample() method then additional CSV files are
created (one per chain) that provide access to quantities used under the
hood by Stan’s implementation of dynamic Hamiltonian Monte Carlo.
CmdStanR does not yet provide a special method for processing these files but they can be read into R using R’s standard CSV reading functions.
fit <- mod$sample(data = data_list, save_latent_dynamics = TRUE)
fit$latent_dynamics_files()[1] "/var/folders/s0/zfzm55px2nd2v__zlw5xfj2h0000gn/T/RtmpXOnqh8/bernoulli-diagnostic-202312131009-1-425883.csv"
[2] "/var/folders/s0/zfzm55px2nd2v__zlw5xfj2h0000gn/T/RtmpXOnqh8/bernoulli-diagnostic-202312131009-2-425883.csv"
[3] "/var/folders/s0/zfzm55px2nd2v__zlw5xfj2h0000gn/T/RtmpXOnqh8/bernoulli-diagnostic-202312131009-3-425883.csv"
[4] "/var/folders/s0/zfzm55px2nd2v__zlw5xfj2h0000gn/T/RtmpXOnqh8/bernoulli-diagnostic-202312131009-4-425883.csv"
# read one of the files in
x <- utils::read.csv(fit$latent_dynamics_files()[1], comment.char = "#")
head(x) lp__ accept_stat__ stepsize__ treedepth__ n_leapfrog__ divergent__
1 -9.73111 1.000000 0.932232 2 3 0
2 -6.75346 1.000000 0.932232 1 3 0
3 -8.48107 0.665681 0.932232 1 3 0
4 -7.74234 1.000000 0.932232 1 1 0
5 -6.82333 1.000000 0.932232 2 3 0
6 -6.82333 0.784555 0.932232 1 3 0
energy__ theta p_theta g_theta
1 10.97150 0.3939970 -2.155610 4.166930
2 9.05119 -1.0295100 -2.933880 0.158161
3 8.61284 0.0530292 0.702591 3.159050
4 8.45793 -0.2148440 1.637290 2.357940
5 7.76204 -1.3633100 1.875250 -0.555575
6 8.04989 -1.3633100 -2.143570 -0.555575The column lp__ is also provided via
fit$draws(), and the columns accept_stat__,
stepsize__, treedepth__,
n_leapfrog__, divergent__, and
energy__ are also provided by
fit$sampler_diagnostics(), but there are several columns
unique to the latent dynamics file.
theta p_theta g_theta
1 0.3939970 -2.155610 4.166930
2 -1.0295100 -2.933880 0.158161
3 0.0530292 0.702591 3.159050
4 -0.2148440 1.637290 2.357940
5 -1.3633100 1.875250 -0.555575
6 -1.3633100 -2.143570 -0.555575Our model has a single parameter theta and the three
columns above correspond to theta in the
unconstrained space (theta on the constrained
space is accessed via fit$draws()), the auxiliary momentum
p_theta, and the gradient g_theta. In general,
each of these three columns will exist for every parameter in
the model.
CmdStanR can of course be used for developing other packages that require compiling and running Stan models as well as using new or custom Stan features available through CmdStan.
You may compile a Stan model at runtime (e.g. just before sampling), or you may compile all the models inside the package file system in advance at installation time. The latter avoids compilations at runtime, which matters in centrally managed R installations where users should not compile their own software.
To pre-compile all the models in a package, you may create top-level
scripts configure and configure.win which run
cmdstan_model() with compile = TRUE and save
the compiled executables somewhere inside the inst/ folder
of the package source. The instantiate
package helps developers configure packages this way, and it documents
other topics such as submitting to CRAN and administering CmdStan. Kevin
Ushey’s configure
package helps create and manage package configuration files in
general.
When developing or testing new features it might be useful to have
more information on how CmdStan is called internally and to see more
information printed when compiling or running models. This can be
enabled for an entire R session by setting the option
"cmdstanr_verbose" to TRUE.
options("cmdstanr_verbose"=TRUE)
mod <- cmdstan_model(stan_file, force_recompile = TRUE)Running make \
/var/folders/s0/zfzm55px2nd2v__zlw5xfj2h0000gn/T/RtmpXOnqh8/model-742871ce8467 \
"STANCFLAGS += --name='bernoulli_model'"
--- Translating Stan model to C++ code ---
bin/stanc --name='bernoulli_model' --o=/var/folders/s0/zfzm55px2nd2v__zlw5xfj2h0000gn/T/RtmpXOnqh8/model-742871ce8467.hpp /var/folders/s0/zfzm55px2nd2v__zlw5xfj2h0000gn/T/RtmpXOnqh8/model-742871ce8467.stan
--- Compiling, linking C++ code ---
clang++ -Wno-deprecated-declarations -Wno-deprecated-declarations -std=c++1y -Wno-unknown-warning-option -Wno-tautological-compare -Wno-sign-compare -D_REENTRANT -Wno-ignored-attributes -I stan/lib/stan_math/lib/tbb_2020.3/include -O3 -I src -I stan/src -I stan/lib/rapidjson_1.1.0/ -I lib/CLI11-1.9.1/ -I stan/lib/stan_math/ -I stan/lib/stan_math/lib/eigen_3.4.0 -I stan/lib/stan_math/lib/boost_1.78.0 -I stan/lib/stan_math/lib/sundials_6.1.1/include -I stan/lib/stan_math/lib/sundials_6.1.1/src/sundials -DBOOST_DISABLE_ASSERTS -c -include-pch stan/src/stan/model/model_header_13_0.hpp.gch -x c++ -o /var/folders/s0/zfzm55px2nd2v__zlw5xfj2h0000gn/T/RtmpXOnqh8/model-742871ce8467.o /var/folders/s0/zfzm55px2nd2v__zlw5xfj2h0000gn/T/RtmpXOnqh8/model-742871ce8467.hpp
clang++ -Wno-deprecated-declarations -Wno-deprecated-declarations -std=c++1y -Wno-unknown-warning-option -Wno-tautological-compare -Wno-sign-compare -D_REENTRANT -Wno-ignored-attributes -I stan/lib/stan_math/lib/tbb_2020.3/include -O3 -I src -I stan/src -I stan/lib/rapidjson_1.1.0/ -I lib/CLI11-1.9.1/ -I stan/lib/stan_math/ -I stan/lib/stan_math/lib/eigen_3.4.0 -I stan/lib/stan_math/lib/boost_1.78.0 -I stan/lib/stan_math/lib/sundials_6.1.1/include -I stan/lib/stan_math/lib/sundials_6.1.1/src/sundials -DBOOST_DISABLE_ASSERTS -Wl,-L,"/Users/jgabry/.cmdstan/cmdstan-2.33.1/stan/lib/stan_math/lib/tbb" -Wl,-rpath,"/Users/jgabry/.cmdstan/cmdstan-2.33.1/stan/lib/stan_math/lib/tbb" /var/folders/s0/zfzm55px2nd2v__zlw5xfj2h0000gn/T/RtmpXOnqh8/model-742871ce8467.o src/cmdstan/main.o -Wl,-L,"/Users/jgabry/.cmdstan/cmdstan-2.33.1/stan/lib/stan_math/lib/tbb" -Wl,-rpath,"/Users/jgabry/.cmdstan/cmdstan-2.33.1/stan/lib/stan_math/lib/tbb" stan/lib/stan_math/lib/sundials_6.1.1/lib/libsundials_nvecserial.a stan/lib/stan_math/lib/sundials_6.1.1/lib/libsundials_cvodes.a stan/lib/stan_math/lib/sundials_6.1.1/lib/libsundials_idas.a stan/lib/stan_math/lib/sundials_6.1.1/lib/libsundials_kinsol.a stan/lib/stan_math/lib/tbb/libtbb.dylib stan/lib/stan_math/lib/tbb/libtbbmalloc.dylib stan/lib/stan_math/lib/tbb/libtbbmalloc_proxy.dylib -o /var/folders/s0/zfzm55px2nd2v__zlw5xfj2h0000gn/T/RtmpXOnqh8/model-742871ce8467
rm -f /var/folders/s0/zfzm55px2nd2v__zlw5xfj2h0000gn/T/RtmpXOnqh8/model-742871ce8467.o
fit <- mod$sample(
data = data_list,
chains = 1,
iter_warmup = 100,
iter_sampling = 100
)Running MCMC with 1 chain...
Running ./bernoulli 'id=1' random 'seed=90958316' data \
'file=/var/folders/s0/zfzm55px2nd2v__zlw5xfj2h0000gn/T/RtmpXOnqh8/standata-742874baeb02.json' \
output \
'file=/var/folders/s0/zfzm55px2nd2v__zlw5xfj2h0000gn/T/RtmpXOnqh8/bernoulli-202312131009-1-4db120.csv' \
'profile_file=/var/folders/s0/zfzm55px2nd2v__zlw5xfj2h0000gn/T/RtmpXOnqh8/bernoulli-profile-202312131009-1-1c1f83.csv' \
'method=sample' 'num_samples=100' 'num_warmup=100' 'save_warmup=0' \
'algorithm=hmc' 'engine=nuts' adapt 'engaged=1'
Chain 1 method = sample (Default)
Chain 1 sample
Chain 1 num_samples = 100
Chain 1 num_warmup = 100
Chain 1 save_warmup = 0 (Default)
Chain 1 thin = 1 (Default)
Chain 1 adapt
Chain 1 engaged = 1 (Default)
Chain 1 gamma = 0.050000000000000003 (Default)
Chain 1 delta = 0.80000000000000004 (Default)
Chain 1 kappa = 0.75 (Default)
Chain 1 t0 = 10 (Default)
Chain 1 init_buffer = 75 (Default)
Chain 1 term_buffer = 50 (Default)
Chain 1 window = 25 (Default)
Chain 1 algorithm = hmc (Default)
Chain 1 hmc
Chain 1 engine = nuts (Default)
Chain 1 nuts
Chain 1 max_depth = 10 (Default)
Chain 1 metric = diag_e (Default)
Chain 1 metric_file = (Default)
Chain 1 stepsize = 1 (Default)
Chain 1 stepsize_jitter = 0 (Default)
Chain 1 num_chains = 1 (Default)
Chain 1 id = 1 (Default)
Chain 1 data
Chain 1 file = /var/folders/s0/zfzm55px2nd2v__zlw5xfj2h0000gn/T/RtmpXOnqh8/standata-742874baeb02.json
Chain 1 init = 2 (Default)
Chain 1 random
Chain 1 seed = 90958316
Chain 1 output
Chain 1 file = /var/folders/s0/zfzm55px2nd2v__zlw5xfj2h0000gn/T/RtmpXOnqh8/bernoulli-202312131009-1-4db120.csv
Chain 1 diagnostic_file = (Default)
Chain 1 refresh = 100 (Default)
Chain 1 sig_figs = -1 (Default)
Chain 1 profile_file = /var/folders/s0/zfzm55px2nd2v__zlw5xfj2h0000gn/T/RtmpXOnqh8/bernoulli-profile-202312131009-1-1c1f83.csv
Chain 1 num_threads = 1 (Default)
Chain 1 Gradient evaluation took 5e-06 seconds
Chain 1 1000 transitions using 10 leapfrog steps per transition would take 0.05 seconds.
Chain 1 Adjust your expectations accordingly!
Chain 1 WARNING: There aren't enough warmup iterations to fit the
Chain 1 three stages of adaptation as currently configured.
Chain 1 Reducing each adaptation stage to 15%/75%/10% of
Chain 1 the given number of warmup iterations:
Chain 1 init_buffer = 15
Chain 1 adapt_window = 75
Chain 1 term_buffer = 10
Chain 1 Iteration: 1 / 200 [ 0%] (Warmup)
Chain 1 Iteration: 100 / 200 [ 50%] (Warmup)
Chain 1 Iteration: 101 / 200 [ 50%] (Sampling)
Chain 1 Iteration: 200 / 200 [100%] (Sampling)
Chain 1 Elapsed Time: 0 seconds (Warm-up)
Chain 1 0.001 seconds (Sampling)
Chain 1 0.001 seconds (Total)
Chain 1 finished in 0.0 seconds.