Initializing the NUTS-HMC sampler#

In this example, we show how to use parameter estimates returned by any of Stan’s inference algorithms as initial values for Stan’s NUTS-HMC sampler. These include:

By default, the NUTS-HMC sampler randomly initializes all (unconstrained) model parameters uniformly in the interval (-2, 2). If this interval is far from the typical set of the posterior, initializing sampling from these approximation algorithms can speed up and improve adaptation.

Model and data#

The Stan model and data are taken from the posteriordb package.

We use the blr model, a Bayesian standard linear regression model with noninformative priors, and its corresponding simulated dataset sblri.json, which was simulated via script sblr.R. For convenience, this example assumes the posteriordb model and data are local, in files blr.stan and sblri.json.

[1]:
import os
from cmdstanpy import CmdStanModel

stan_file = 'blr.stan' # basic linear regression
data_file = 'sblri.json' # simulated data

model = CmdStanModel(stan_file=stan_file)

print(model.code())
data {
  int<lower=0> N;
  int<lower=0> D;
  matrix[N, D] X;
  vector[N] y;
}
parameters {
  vector[D] beta;
  real<lower=0> sigma;
}
model {
  // prior
  target += normal_lpdf(beta | 0, 10);
  target += normal_lpdf(sigma | 0, 10);
  // likelihood
  target += normal_lpdf(y | X * beta, sigma);
}


Demonstration with Stan’s pathfinder method#

Initializing the sampler with estimates from any previous inference algorithm follows the same general usage pattern. First, we call the corresponding method on the CmdStanModel object. From the resulting fit, we call the .create_inits() method to construct a set of per-chain initializations for the model parameters. To make it explicit, we will walk through the process using the pathfinder method (which wraps the CmdStan pathfinder method).

Pathfinder locates normal approximations to the target density along a quasi-Newton optimization path, with local covariance estimated using the negative inverse Hessian estimates produced by the LBFGS optimizer. Pathfinder returns draws from the Gaussian approximation with the lowest estimated Kullback-Leibler (KL) divergence to the true posterior. By default, CmdStanPy runs multi-path Pathfinder which returns an importance-resampled set of draws over the outputs of 4 independent single-path Pathfinders. This better matches non-normal target densities and also mitigates the problem of L-BFGS getting stuck at local optima or in saddle points on plateaus.

We obtain Pathfinder estimates by calling the .pathfinder() method which returns a CmdStanPathfinder object:

[2]:
pathfinder_fit = model.pathfinder(data=data_file, seed=123)

Posteriordb provides reference posteriors for all models. For the blr model, conditioned on the dataset sblri.json, the reference posteriors can be found in the sblri-blr.json file.

The reference posteriors for all elements of beta and sigma are all very close to 1.0.

The experiments reported in Figure 3 of the paper Pathfinder: Parallel quasi-Newton variational inference by Zhang et al. show that Pathfinder provides a better estimate of the posterior, as measured by the 1-Wasserstein distance to the reference posterior, than 75 iterations of the warmup Phase I algorithm used by the NUTS-HMC sampler. Furthermore, Pathfinder is more computationally efficient, requiring fewer evaluations of the log density and gradient functions. Therefore, using the Pathfinder estimates to initialize the parameter values for the NUTS-HMC sampler can allow the sampler to do a better job of adapting the stepsize and metric during warmup, resulting in better performance and estimation.

We construct the parameter inits for full MCMC sampling below. The .create_inits() default behavior is to create inits for four chains to correspond with the sampling defaults. You can requests more or less by modifying the chains keyword argument.

[3]:
pathfinder_inits = pathfinder_fit.create_inits()
for chain_init in pathfinder_inits:
    print(chain_init)
{'beta': array([0.99856073, 1.0004443 , 1.0008198 , 1.0009798 , 1.0029835 ]), 'sigma': array(0.91601066)}
{'beta': array([0.99929719, 1.0005749 , 1.000647  , 1.0007698 , 1.0018447 ]), 'sigma': array(0.98492136)}
{'beta': array([0.99849737, 1.0019589 , 1.0001937 , 1.0024171 , 1.001133  ]), 'sigma': array(0.92986127)}
{'beta': array([0.99939598, 1.0011405 , 0.99885316, 1.0020774 , 1.0009952 ]), 'sigma': array(0.98083425)}

We see that the Pathfinder inits are close the reference posteriors for the parameters. To use these inits, we pass the pathfinder_inits object to the inits kwarg:

[4]:
mcmc_pathfinder_inits_fit = model.sample(
    data=data_file, inits=pathfinder_inits, iter_warmup=75, seed=12345
)

[5]:
print(mcmc_pathfinder_inits_fit.diagnose())
Checking sampler transitions treedepth.
Treedepth satisfactory for all transitions.

Checking sampler transitions for divergences.
No divergent transitions found.

Checking E-BFMI - sampler transitions HMC potential energy.
E-BFMI satisfactory.

Rank-normalized split effective sample size satisfactory for all parameters.

The following parameters had rank-normalized split R-hat greater than 1.01:
  sigma
Such high values indicate incomplete mixing and biased estimation.
You should consider regularizing your model with additional prior information or a more effective parameterization.

Processing complete.

Despite only running 75 warmup iterations, all posterior diagnostics from the sampler look good.

[6]:
mcmc_pathfinder_inits_fit.summary()
[6]:
Mean MCSE StdDev MAD 5% 50% 95% ESS_bulk ESS_tail ESS_bulk/s R_hat
lp__ -156.878000 0.059274 1.803860 1.595760 -160.306000 -156.507000 -154.66400 969.251 1539.550 1926.940 1.00525
beta[1] 0.999448 0.000014 0.000945 0.000934 0.997882 0.999457 1.00104 4703.190 3138.750 9350.270 1.00073
beta[2] 1.000240 0.000018 0.001173 0.001140 0.998356 1.000210 1.00218 4193.590 3103.830 8337.150 1.00239
beta[3] 1.000440 0.000013 0.000946 0.000927 0.998894 1.000450 1.00199 5095.050 3366.110 10129.300 1.00105
beta[4] 1.001120 0.000015 0.001054 0.001024 0.999377 1.001120 1.00288 4714.660 3069.910 9373.090 1.00152
beta[5] 1.001550 0.000015 0.001054 0.001025 0.999828 1.001530 1.00335 4693.550 3416.820 9331.120 1.00163
sigma 0.963636 0.004620 0.070141 0.067402 0.849318 0.960596 1.08481 229.339 330.003 455.941 1.02149

If we were to instead use the default random parameter initializations, we would need to run more warmup iterations to produce useful samples. For example, if we only run the same 75 warmup iterations with random inits, the result fails to estimate sigma correctly:

[7]:
mcmc_random_inits_fit = model.sample(data=data_file, iter_warmup=75, seed=12345)

17:12:53 - cmdstanpy - WARNING - Some chains may have failed to converge.
        Chain 1 had 10 divergent transitions (1.0%)
        Chain 3 had 184 divergent transitions (18.4%)
        Chain 4 had 116 divergent transitions (11.6%)
        Use the "diagnose()" method on the CmdStanMCMC object to see further information.

[8]:
mcmc_random_inits_fit.summary()
[8]:
Mean MCSE StdDev MAD 5% 50% 95% ESS_bulk ESS_tail ESS_bulk/s R_hat
lp__ -186.134000 24.068300 34.291600 28.859000 -242.280000 -175.92900 -154.92000 5.04762 2000.0000 25.2381 2.28409
beta[1] 0.999572 0.000199 0.002070 0.001585 0.996770 0.99944 1.00418 124.68800 128.6520 623.4410 1.24174
beta[2] 1.000040 0.000255 0.002465 0.001865 0.995116 1.00017 1.00371 122.95100 2000.0000 614.7570 1.16596
beta[3] 1.000520 0.000188 0.002244 0.001550 0.996533 1.00043 1.00483 194.82000 2000.0000 974.0980 1.16676
beta[4] 1.001160 0.000173 0.002213 0.001479 0.997934 1.00110 1.00508 214.09600 64.9385 1070.4800 1.08473
beta[5] 1.001510 0.000150 0.002168 0.001510 0.997302 1.00140 1.00545 239.97100 2000.0000 1199.8500 1.13135
sigma 1.839250 0.718889 1.027200 0.816670 0.892553 1.52204 3.54603 5.04357 2000.0000 25.2178 2.38890
[9]:
print(mcmc_random_inits_fit.diagnose())
Checking sampler transitions treedepth.
Treedepth satisfactory for all transitions.

Checking sampler transitions for divergences.
310 of 4000 (7.75%) transitions ended with a divergence.
These divergent transitions indicate that HMC is not fully able to explore the posterior distribution.
Try increasing adapt delta closer to 1.
If this doesn't remove all divergences, try to reparameterize the model.

Checking E-BFMI - sampler transitions HMC potential energy.
The E-BFMI, 0.01, is below the nominal threshold of 0.30 which suggests that HMC may have trouble exploring the target distribution.
If possible, try to reparameterize the model.

Rank-normalized split effective sample size satisfactory for all parameters.

The following parameters had rank-normalized split R-hat greater than 1.01:
  beta[1], beta[2], beta[3], beta[4], beta[5], sigma
Such high values indicate incomplete mixing and biased estimation.
You should consider regularizing your model with additional prior information or a more effective parameterization.

Processing complete.

The diagnostics clearly indicate problems with estimating sigma. In this case, it is necessary to run the model with at least 150 warmup iterations to produce a good set of estimates when starting from the default initialization.

Other inference algorithms#

We can follow the same pattern with Stan’s ADVI algorithm by first using the CmdStanModel.variationl method. Because this algorithm is unstable and may fail to converge, we run it with argument require_converged set to False. We also specify a seed, to avoid instabilities as well as for reproducibility.

[10]:
vb_fit = model.variational(data=data_file, require_converged=False, seed=123)
17:12:53 - cmdstanpy - WARNING - The algorithm may not have converged.
Proceeding because require_converged is set to False

The ADVI algorithm provides estimates of all model parameters.

The variational method returns a CmdStanVB object, which similarly can construct a set of inits with the .create_inits() method:

[11]:
vb_inits = vb_fit.create_inits()
for chain_init in vb_inits:
    print(chain_init)
{'beta': array([1.0064444 , 1.0010619 , 1.0128921 , 0.99894687, 0.98802581]), 'sigma': array(1.5788509)}
{'beta': array([1.0069349 , 1.0004313 , 1.0094822 , 0.99908589, 0.9901926 ]), 'sigma': array(1.6325343)}
{'beta': array([1.0076668 , 1.0028412 , 1.0139674 , 0.99754085, 0.98745047]), 'sigma': array(1.6172484)}
{'beta': array([1.0066871 , 1.0016641 , 1.006235  , 0.99546738, 0.9914063 ]), 'sigma': array(1.6427579)}

Which can be passed to the inits keyword argument of the sample method.

[12]:
mcmc_vb_inits_fit = model.sample(
    data=data_file, inits=vb_inits, iter_warmup=75, seed=12345
)

17:12:53 - cmdstanpy - WARNING - Non-fatal error during sampling:
Exception: normal_lpdf: Scale parameter is 0, but must be positive! (in 'blr.stan', line 16, column 2 to column 45)
Exception: normal_lpdf: Scale parameter is 0, but must be positive! (in 'blr.stan', line 16, column 2 to column 45)
Exception: normal_lpdf: Scale parameter is 0, but must be positive! (in 'blr.stan', line 16, column 2 to column 45)
Exception: normal_lpdf: Scale parameter is 0, but must be positive! (in 'blr.stan', line 16, column 2 to column 45)
Consider re-running with show_console=True if the above output is unclear!

[13]:
print(mcmc_vb_inits_fit.diagnose())
Checking sampler transitions treedepth.
Treedepth satisfactory for all transitions.

Checking sampler transitions for divergences.
No divergent transitions found.

Checking E-BFMI - sampler transitions HMC potential energy.
E-BFMI satisfactory.

Rank-normalized split effective sample size satisfactory for all parameters.

Rank-normalized split R-hat values satisfactory for all parameters.

Processing complete, no problems detected.

[14]:
mcmc_vb_inits_fit.summary()
[14]:
Mean MCSE StdDev MAD 5% 50% 95% ESS_bulk ESS_tail ESS_bulk/s R_hat
lp__ -156.804000 0.047414 1.659270 1.582390 -159.918000 -156.514000 -154.62600 1310.150 2529.210 2957.460 1.003470
beta[1] 0.999455 0.000015 0.000981 0.000962 0.997824 0.999457 1.00109 4487.250 3352.000 10129.200 1.000560
beta[2] 1.000250 0.000019 0.001156 0.001165 0.998411 1.000240 1.00215 3703.710 2480.130 8360.520 1.002550
beta[3] 1.000430 0.000013 0.000937 0.000922 0.998873 1.000420 1.00196 5057.080 3609.230 11415.500 1.001490
beta[4] 1.001140 0.000016 0.001060 0.001061 0.999399 1.001130 1.00288 4470.820 3496.380 10092.100 1.000530
beta[5] 1.001560 0.000015 0.001075 0.001087 0.999799 1.001550 1.00335 4843.510 2617.610 10933.400 0.999641
sigma 0.957700 0.003962 0.061501 0.065358 0.861220 0.956082 1.06582 250.391 292.454 565.218 1.014810

The sampler estimates match the reference posterior with no diagnostic issues.

Inits can also be constructed from the laplace method:

[15]:
laplace_inits = model.laplace_sample(data=data_file, seed=123).create_inits()
[16]:
mcmc_laplace_inits_fit = model.sample(
    data=data_file, inits=laplace_inits, iter_warmup=75, seed=12345
)

[17]:
print(mcmc_laplace_inits_fit.diagnose())
Checking sampler transitions treedepth.
Treedepth satisfactory for all transitions.

Checking sampler transitions for divergences.
No divergent transitions found.

Checking E-BFMI - sampler transitions HMC potential energy.
E-BFMI satisfactory.

Rank-normalized split effective sample size satisfactory for all parameters.

Rank-normalized split R-hat values satisfactory for all parameters.

Processing complete, no problems detected.

And the optimize method. Since optimizations attempts to return a posterior mode, this will initialize all chains at the same point, which is not typically ideal.

[18]:
optimized_inits = model.optimize(data=data_file, seed=123).create_inits()
[19]:
mcmc_optimize_inits_fit = model.sample(
    data=data_file, inits=optimized_inits, iter_warmup=75, seed=12345
)

[20]:
print(mcmc_optimize_inits_fit.diagnose())
Checking sampler transitions treedepth.
Treedepth satisfactory for all transitions.

Checking sampler transitions for divergences.
No divergent transitions found.

Checking E-BFMI - sampler transitions HMC potential energy.
E-BFMI satisfactory.

Rank-normalized split effective sample size satisfactory for all parameters.

The following parameters had rank-normalized split R-hat greater than 1.01:
  sigma
Such high values indicate incomplete mixing and biased estimation.
You should consider regularizing your model with additional prior information or a more effective parameterization.

Processing complete.

It is also possible to use the output of the sample() method itself to construct inits to be fed into a future sampling run:

[21]:
first_mcmc_inits = model.sample(
    data=data_file, iter_warmup=75, seed=12345
).create_inits()

17:12:54 - cmdstanpy - WARNING - Some chains may have failed to converge.
        Chain 1 had 10 divergent transitions (1.0%)
        Chain 3 had 184 divergent transitions (18.4%)
        Chain 4 had 116 divergent transitions (11.6%)
        Use the "diagnose()" method on the CmdStanMCMC object to see further information.

[22]:
second_mcmc_fit = model.sample(data=data_file, inits=first_mcmc_inits, iter_warmup=75, seed=12345)

17:12:54 - cmdstanpy - WARNING - Non-fatal error during sampling:
Exception: normal_lpdf: Scale parameter is 0, but must be positive! (in 'blr.stan', line 16, column 2 to column 45)
Exception: normal_lpdf: Scale parameter is 0, but must be positive! (in 'blr.stan', line 16, column 2 to column 45)
Exception: normal_lpdf: Scale parameter is 0, but must be positive! (in 'blr.stan', line 16, column 2 to column 45)
Consider re-running with show_console=True if the above output is unclear!

[23]:
print(second_mcmc_fit.diagnose())
Checking sampler transitions treedepth.
Treedepth satisfactory for all transitions.

Checking sampler transitions for divergences.
No divergent transitions found.

Checking E-BFMI - sampler transitions HMC potential energy.
E-BFMI satisfactory.

Rank-normalized split effective sample size satisfactory for all parameters.

Rank-normalized split R-hat values satisfactory for all parameters.

Processing complete, no problems detected.

We see that despite the initial sampling issues in the first MCMC run, the inits sourced from that run result in reasonable sampling in the second run.