In this section, we will explore several interesting variants of BERT. We will learn about popular variants of BERT such as ALBERT, RoBERTa, ELECTRA, and SpanBERT. We will also explore BERT variants based on knowledge distillation, such as DistilBERT and TinyBERT. 

The following chapters are included in this section:

• Chapter 4, BERT Variants I – ALBERT, RoBERTa, ELECTRA, and SpanBERT
• Chapter 5, BERT Variants II – Based on Knowledge Distillation