Simulation and synthesis are core parts of the future of AI and machine learning. Consider: programmers, data scientists, and machine learning engineers can create the brain of a self-driving car without the car. Rather than use information from the real world, you can create artificial data using simulations to train traditional machine learning models. That's just the beginning.

With this practical book, you'll explore the possibilities of simulation- and synthesis-based machine learning and AI, with a focus on deep reinforcement learning and imitation learning techniques. AI and ML are increasingly data driven, and simulations are a powerful, engaging way to unlock their full potential.

With this deeply practical book, you'll learn how to:

  • Design an approach for solving ML and AI problems using simulations
  • Use a game engine to synthesize images for use as training data
  • Create simulation environments designed for training deep reinforcement learning and imitation learning
  • Use and apply efficient general-purpose algorithms for simulation-based ML, such as proximal policy optimization (PPO) and soft actor-critic (SAO)
  • Train ML models locally, concurrently, and in the cloud
  • Use PyTorch, TensorFlow, the Unity ML-Agents and Perception Toolkits to enable ML tools to work with industry-standard game development tools

Table of Contents

  1. Preface
    1. Resources Used in This Book
    2. Audience and Approach
    3. Organization of This Book
    4. Using This Book
    5. Our Tasks
    6. Conventions Used in This Book
    7. Using Code Examples
    8. O’Reilly Online Learning
    9. How to Contact Us
    10. Acknowledgments
  2. 1. Introducing Synthesis and Simulation
    1. A whole new world of ML
    2. The Domains
    3. Simulation
    4. Synthesis
    5. The Tools
    6. Unity
    7. TensorFlow and Unity ML-Agents
    8. Unity ML-Agents
    9. The Techniques
    10. Reinforcement Learning
    11. Imitation Learning
    12. Summary of Techniques
    13. Projects
    14. Simulation Projects
    15. Synthesis Projects
    16. Summary and Next Steps
  3. 2. Creating Your First Simulation
    1. Our Simulation
    2. Setting up
    3. Creating the Unity Project
    4. The Environment
    5. The Floor
    6. The Target
    7. The Agent
    8. Starting and Stopping the Agent
    9. Letting the agent observe the environment
    10. Letting the agent take actions in the environment
    11. Giving the agent rewards for its behaviour
    12. Finishing touches for an agent
    13. Providing a manual control system for the agent
    14. Training with the Simulation
    15. Monitoring the training with TensorBoard
    16. When the training is complete
    17. What’s it all mean?
    18. Coming up next
  4. 3. Creating Your First Synthesised Data
    1. Unity Perception
    2. The Process
    3. Using Unity Perception
    4. Creating the Unity Project
    5. Creating a Scene
    6. Getting the Dice Models
    7. A Very Simple Scene
    8. Preparing for Synthesis
    9. Test the Scenario
    10. What’s next?