The architecture for my Lunar Lander agent is only slightly more complicated than for CartPole, introducing just a few more neurons for the same three hidden layers. We will use the following code to define the model:

def build_model(state_size, num_actions):
    input = Input(shape=(1, state_size))
    x = Flatten()(input)
    x = Dense(64, activation='relu')(x)
    x = Dense(32, activation='relu')(x)
    x = Dense(16, activation='relu')(x)
    output = Dense(num_actions, activation='linear')(x)
    model = Model(inputs=input, outputs=output)
    print(model.summary())
    return model

In the case of this problem, smaller architectures resulted in an agent that learned to control and hover the lander, but not actually land it. Of course, because we're making minibatch updates for every step in every episode, we need to carefully weigh complexity against runtime and computational needs.