Parallel calls for map transformations

Any map transformation you call on your dataset by default acts only on a single element of your dataset, and it will process elements sequentially. The easiest thing you can do to speed things up and use all your CPU power is to set the num_parallel_calls argument to the number of CPU cores you have available to you. This way, we don't waste any CPU power available to us. You are warned, however, not to set this higher than the number of cores available to you, as this may actually reduce performance because of inefficient scheduling.

Any transformations you want to do to your data, such as data augmentations, can also be written as functions and then passed to the map method as before, to apply them to the dataset. For example, note the following code:

train_dataset = train_dataset.map(decode_tfrec, num_parallel_calls=4)  # Decode tfrecord.
train_dataset = train_dataset.map(data_augmentation,
num_parallel_calls=4) # Augment data.
..................Content has been hidden....................

You can't read the all page of ebook, please click here login for view all page.
Reset
18.191.254.44