We'll also need to know about perception, of course! In order to interact with any object in the environment, we first need to visualize it. We need to know where it is and what it is, and that's what perception is for!

Perception is usually done using RGBD cameras such as Kinect, which is used to improve manipulation tasks such as detecting new objects that spawn into the simulation.