An important piece of information about gym-gazebo is that the repository has been archived and is not supported anymore. However, the same is tested in Ubuntu 18.04 and is working fine with ROS-Melodic. It should act as a good start to understand reinforcement learning implementation in ROS.

The redesigned and upgraded version of gym-gazebo is the gym-gazebo2 extension by Acutronic robotics. This new version discusses the new ROS-2 based architecture and summarizes results using another reinforcement learning technique called Proximal Policy Optimization (PPO). The authors have tested this with a ROS-2 compliant robot arm called MARA (short for Modular Articulated Robot ARM) and have achieved accuracies of millimeter scale while achieving kinematics. More information on this extension can be found in this paper: https://arxiv.org/pdf/1903.06278.pdf. Currently, there is only one robot-environment pair has been implemented and tested, so let's see how this example works. The next section introduces MARA and its environment.