Sequence-to-sequence models can be built at either the character level or the word level. A word-level sequence-to-sequence model will take words as the atomic unit of the input, and a character-level model will take characters as the atomic unit of the input.

So, which should you use? Typically, the best results are obtained from word-level models. That said, predicting the most probable next word in a sequence requires a softmax layer as wide as the vocabulary of the problem. This results in a very wide, highly dimensional problem.

Character-level models are much smaller. There are 26 letters in the alphabet but there are about 171,000 English words in common use.

For the problem we present in this chapter, I'll use a character-level model because I value your AWS budget. Converting to words is fairly straightforward, with the majority of the complexity being in the data prep, which is an exercise left to the reader.