The Storage tab shows the size and memory use for each RDD, DataFrame, or Dataset. You should be able to see the storage-related information of RDDs, DataFrames, or Datasets. The following figure shows storage metadata such as RDD name, storage level, the number of cache partitions, the percentage of a fraction of the data that was cached, and the size of the RDD in the main memory:
Figure 9: Storage tab shows space consumed by an RDD in disk
Note that if the RDD cannot be cached in the main memory, disk space will be used instead. A more detailed discussion will be carried out in a later section of this chapter.
Figure 10: Data distribution and the storage used by the RDD in disk