The $loo() method computes approximate LOO-CV using the
loo package. In order to use this method you must compute and save
the pointwise log-likelihood in your Stan program. See loo::loo.array()
and the loo package vignettes
for details.
loo(variables = "log_lik", r_eff = TRUE, moment_match = FALSE, ...)(character vector) The name(s) of the variable(s) in the
Stan program containing the pointwise log-likelihood. The default is to
look for "log_lik". This argument is passed to the
$draws() method.
(multiple options) How to handle the r_eff argument for loo():
TRUE (the default) will automatically call loo::relative_eff.array()
to compute the r_eff argument to pass to loo::loo.array().
FALSE or NULL will avoid computing r_eff (which can sometimes be slow),
but the reported ESS and MCSE estimates can be over-optimistic if the
posterior draws are not (near) independent.
If r_eff is anything else, that object will be passed as the r_eff
argument to loo::loo.array().
(logical) Whether to use a
moment-matching correction for problematic
observations. The default is FALSE. Using moment_match=TRUE will result
in compiling the additional methods described in
fit-method-init_model_methods. This allows CmdStanR to automatically
supply the functions for the log_lik_i, unconstrain_pars,
log_prob_upars, and log_lik_i_upars arguments to
loo::loo_moment_match().
Other arguments (e.g., cores, save_psis, etc.) passed to
loo::loo.array() or loo::loo_moment_match.default()
(if moment_match = TRUE is set).
The object returned by loo::loo.array() or
loo::loo_moment_match.default().
The loo package website with documentation and vignettes.
# \dontrun{
# the "logistic" example model has "log_lik" in generated quantities
fit <- cmdstanr_example("logistic")
loo_result <- fit$loo(cores = 2)
print(loo_result)
#>
#> Computed from 4000 by 100 log-likelihood matrix.
#>
#> Estimate SE
#> elpd_loo -63.7 4.1
#> p_loo 3.9 0.5
#> looic 127.4 8.2
#> ------
#> MCSE of elpd_loo is 0.0.
#> MCSE and ESS estimates assume MCMC draws (r_eff in [0.9, 1.4]).
#>
#> All Pareto k estimates are good (k < 0.7).
#> See help('pareto-k-diagnostic') for details.
# }