Let's visualize our network in TensorBoard. To launch TensorBoard, open your Terminal and type the following: 

tensorboard --logdir=logs --port=6007 --host=127.0.0.1

This is our A3C network. We have one global network and four workers:

Let's expand our global network; you can see we have one actor and one critic:

Okay, what is really going on in workers? Let's expand our worker network. You can see how the worker nodes are performing:

What about the sync node? What is that doing? The sync node pushes the local gradients from the local to the global network and pulls gradients from the global to the local network: