The memory and policy from CartPole can be reused. I believe it might be possible to improve the speed of agent training by further tuning the steps in linear annealed policy because this agent takes many more steps to train. However, the values chosen for CartPole seem to work pretty well enough, so that's an exercise left to the reader.