We will use a slightly more complicated model to illustrate Markov chain Monte Carlo inference:

formula = 'income ~ sex + age+ I(age ** 2) + hours + educ'

Patsy's function, I(), allows us to use regular Python expressions to create new variables on the fly. Here, we square age to capture the non-linear relationship that more experience adds less income later in life.

Note that variables measured on very different scales can slow down the sampling process. Hence, we first apply sklearn's scale() function to standardize the age, hours, and educ variables.

Once we have defined our model with the new formula, we are ready to perform inference to approximate the posterior distribution. MCMC sampling algorithms are available through the pm.sample() function.

By default, PyMC3 automatically selects the most efficient sampler and initializes the sampling process for efficient convergence. For a continuous model, PyMC3 chooses the NUTS sampler that we discussed in the previous section. It also runs variational inference via ADVI to find good starting parameters for the sampler. One among several alternatives is to use the MAP estimate.

To see what convergence looks like, we first draw only 100 samples after tuning the sampler for 1000 iterations. This will be discarded afterwards. The sampling process can be parallelized for multiple chains using the cores argument (except when using GPU):

with logistic_model:
    trace = pm.sample(draws=100, tune=1000,
                      init='adapt_diag', # alternative initialization
                      chains=4, cores=2,
                      random_seed=42)

The resulting trace contains the sampled values for each random variable. We can continue sampling by providing the trace of a prior run as input (see the notebook for more information).