In the SimGAN paper, authors set out to create a refiner network that can accurately improve the realism of synthetic images in an unsupervised manner. In the past, it has been quite hard to find matched simulation and real data for training such networks, but SimGAN has changed the existing landscape thanks to its focus on a simulated and unsupervised architecture. In SimGAN architecture, line-of-sight information is gathered for use in real models that can simulate examples of a similar direction in simulation, which allows the network to recognize a relationship between real and simulated actions. The following diagram illustrates this technique:

To see SimGAN architecture in action, we'll perform the following steps:

1. Build a Docker container and run a script to create models
2. Build a refiner network (the generator code) for the SimGAN architecture
3. Build a discriminator network
4. Develop a training code
5. Evaluate the output of the training code