Compute diagnostics for Pareto smoothing the tail draws of x by replacing tail draws by order statistics of a generalized Pareto distribution fit to the tail(s).
Usage
pareto_diags(x, ...)
# Default S3 method
pareto_diags(
x,
tail = c("both", "right", "left"),
r_eff = NULL,
ndraws_tail = NULL,
verbose = FALSE,
are_log_weights = FALSE,
...
)
# S3 method for class 'rvar'
pareto_diags(x, ..., r_eff = NULL)
pareto_khat_threshold(x, ...)
# Default S3 method
pareto_khat_threshold(x, ...)
# S3 method for class 'rvar'
pareto_khat_threshold(x, ...)
pareto_min_ss(x, ...)
# Default S3 method
pareto_min_ss(x, ...)
# S3 method for class 'rvar'
pareto_min_ss(x, ...)
pareto_convergence_rate(x, ...)
# Default S3 method
pareto_convergence_rate(x, ...)
# S3 method for class 'rvar'
pareto_convergence_rate(x, ...)Arguments
- x
(multiple options) One of:
A matrix of draws for a single variable (iterations x chains). See
extract_variable_matrix().An
rvar.
- ...
Arguments passed to individual methods (if applicable).
- tail
(string) The tail to diagnose/smooth:
"right": diagnose/smooth only the right (upper) tail"left": diagnose/smooth only the left (lower) tail"both": diagnose/smooth both tails and return the maximum k-hat value
The default is
"both".- r_eff
Ignored. A value other than
NULLor 1 gives a warning. Default isNULL.- ndraws_tail
(numeric) number of draws for the tail. If
ndraws_tailis not specified, it is calculated withps_tail_length().- verbose
(logical) Should diagnostic messages be printed? If
TRUE, messages related to Pareto diagnostics will be printed. Default isFALSE.- are_log_weights
(logical) Are the draws log weights? Default is
FALSE. IfTRUEcomputation will take into account that the draws are log weights, and only right tail will be smoothed.
Value
List of Pareto smoothing diagnostics:
khat: estimated Pareto k shape parameter,min_ss: minimum sample size for reliable Pareto smoothed estimate,khat_threshold: khat-threshold for reliable Pareto smoothed estimate,convergence_rate: Pareto smoothed estimate RMSE convergence rate.
Details
When the fitted Generalized Pareto Distribution is used to smooth the tail values and these smoothed values are used to compute expectations, the following diagnostics can give further information about the reliability of these estimates.
min_ss: Minimum sample size for reliable Pareto smoothed estimate. If the actual sample size is greater thanmin_ss, then Pareto smoothed estimates can be considered reliable. If the actual sample size is lower thanmin_ss, increasing the sample size might result in more reliable estimates. For further details, see Section 3.2.3, Equation 11 in Vehtari et al. (2024).khat_threshold: Threshold below which k-hat values result in reliable Pareto smoothed estimates. The threshold is lower for smaller effective sample sizes. If k-hat is larger than the threshold, increasing the total sample size may improve reliability of estimates. For further details, see Section 3.2.4, Equation 13 in Vehtari et al. (2024).convergence_rate: Relative convergence rate compared to the central limit theorem. Applicable only if the actual sample size is sufficiently large (greater thanmin_ss). The convergence rate tells the rate at which the variance of an estimate reduces when the sample size is increased, compared to the central limit theorem convergence rate. See Appendix B in Vehtari et al. (2024).
References
Aki Vehtari, Daniel Simpson, Andrew Gelman, Yuling Yao and Jonah Gabry (2024). Pareto Smoothed Importance Sampling. Journal of Machine Learning Research, 25(72):1-58. PDF
See also
pareto_khat, pareto_min_ss,
pareto_khat_threshold, and pareto_convergence_rate for
individual diagnostics; and pareto_smooth for Pareto smoothing
draws.
Other diagnostics:
ess_basic(),
ess_bulk(),
ess_quantile(),
ess_sd(),
ess_tail(),
mcse_mean(),
mcse_quantile(),
mcse_sd(),
pareto_khat(),
rhat(),
rhat_basic(),
rhat_nested(),
rstar()
Examples
mu <- extract_variable_matrix(example_draws(), "mu")
pareto_diags(mu)
#> $khat
#> [1] 0.2728938
#>
#> $min_ss
#> [1] 23.73094
#>
#> $khat_threshold
#> [1] 0.6156891
#>
#> $convergence_rate
#> [1] 0.970523
#>
d <- as_draws_rvars(example_draws("multi_normal"))
pareto_diags(d$Sigma)
#> $khat
#> [,1] [,2] [,3]
#> [1,] 0.02212907 0.03117354 0.05660144
#> [2,] 0.03117354 0.07103026 0.14702769
#> [3,] 0.05660144 0.14702769 -0.07396071
#>
#> $min_ss
#> [,1] [,2] [,3]
#> [1,] 10.53489 10.76903 11.48147
#> [2,] 10.76903 11.92508 14.87206
#> [3,] 11.48147 14.87206 10.00000
#>
#> $khat_threshold
#> [,1] [,2] [,3]
#> [1,] 0.6156891 0.6156891 0.6156891
#> [2,] 0.6156891 0.6156891 0.6156891
#> [3,] 0.6156891 0.6156891 0.6156891
#>
#> $convergence_rate
#> [,1] [,2] [,3]
#> [1,] 0.9993812 0.9990881 0.9981162
#> [2,] 0.9990881 0.9974527 0.9920776
#> [3,] 0.9981162 0.9920776 1.0000000
#>