The job of the RPN is to predict whether something we call an anchor (essentially just a bounding box) contains an object or is only background, and then to refine the position of this bounding box.

Basically, the RPN does this by sliding a small window (3 x 3) across the last CNN feature map (the same feature map Fast R-CNN gets proposals from). For every sliding window center, we create k fixed anchor boxes and classify those boxes as containing an object or not:

Internally, during training, we choose the anchor bounding box with the biggest IoU with the ground-truth bounding box for backpropagation.