The Bootstrap and Bagging
The bootstrap is a technique for approximately sampling from the error
distribution for an estimator. Thus it can be used as a Monte Carlo
method to estimate standard errors and confidence intervals for point
estimates (Efron and Tibshirani 1986; 1994). It works by
subsampling the original data and computing sample estimates from the
subsample. Like other Monte Carlo methods, the bootstrap is
plug-and-play, allowing great flexibility in both model choice and
Bagging is a technique for combining bootstrapped estimators for model
criticism and more robust inference (Breiman 1996; Huggins and Miller 2019).
Breiman, Leo. 1996. “Bagging Predictors.” Machine Learning 24 (2): 123–40.
Efron, Bradley, and Robert Tibshirani. 1986. “Bootstrap Methods for Standard Errors, Confidence Intervals, and Other Measures of Statistical Accuracy.” Statistical Science 1 (1): 54–75.
Efron, Bradley, and Robert J Tibshirani. 1994. An Introduction to the Bootstrap. Chapman & Hall/CRC.
Huggins, Jonathan H, and Jeffrey W Miller. 2019. “Using Bagged Posteriors for Robust Inference and Model Criticism.” arXiv, no. 1912.07104.