Earlier in the chapter, we talked about an RNN's ability to maintain state, or memory, across time steps.

When using Keras, LSTMs can be configured in two ways, stateful and stateless.

The stateless configuration is the default. When you use a stateless LSTM configuration, LSTM cell memory is reset every batch. This makes batch size a very important consideration. Stateless works best when the the sequences you're learning aren't dependent on one another. Sentence-level prediction of a next word might be a good example of when to use stateless.

The stateful configuration resets LSTM cell memory every epoch. This configuration is most commonly used when each sequence in the training set depends on the sequence that comes before it. If sentence-level prediction might be a good task for a stateless configuration, document-level prediction might be a good task for a stateful model.

Ultimately, this choice is dependent on the problem and may require some experimentation where each option is tested.

For this example, I've tested each option and have chosen to use a stateful model. That's probably not surprising when we consider the context of the problem.