You can also refer to these papers: A3C paper: https://arxiv.org/pdf/1602.01783.pdf Vision enhanced A3C: http://cs231n.stanford.edu/reports/2017/pdfs/617.pdf