The idea of the reparameterization trick is to take out the random sample node from the backpropagation loop. It achieves this by taking a sample epsilon from a Gaussian distribution and then multiplying this by the result of our standard deviation vector σ and then adding μ. The formula for our latent vector is now this:

The produced latent vectors will be the same as before, but making this change now allows the gradients to flow back through to the encoder part of the VAE. The following diagram shows the VAE model before doing reparameterization on the left and after doing it on the right. The blue boxes are the two parts of our loss function. Looking at the diagram, you can see that our gradients can now flow backward, as we no longer have the red box (sample node) blocking the way:

This is what this reparameterization looks like in TensorFlow:

# Add linear ops to produce mean and standard devation vectors
fc_mean = tf.layers.dense(self.__enc_out, units=latent_size, activation=None, name="w_mean") 
fc_stddev = tf.layers.dense(self.__enc_out, units=latent_size, activation=None, name="w_stddev") 
 
# Generate normal distribution with dimensions [Batch, latent_size] 
sample_block = tf.random_normal([tf.shape(X)[0], latent_size], 0, 1, dtype=tf.float32) 
latent_z = fc_mean + (fc_stddev * sample_block)