This is the first step of the training process. In this step, we train the generator and the discriminator networks. Perform the following steps:
- Start by specifying the parameters required for the training:
# Define hyperparameters
data_dir = "/path/to/dataset/directory/"
wiki_dir = os.path.join(data_dir, "wiki_crop")
epochs = 500
batch_size = 128
image_shape = (64, 64, 3)
z_shape = 100
TRAIN_GAN = True
TRAIN_ENCODER = False
TRAIN_GAN_WITH_FR = False
fr_image_shape = (192, 192, 3)
- Next, define the optimizers for the training. We will use the Adam optimizer, which is available in Keras. Initialize the optimizers, as shown in the following code:
# Define optimizers
# Optimizer for the discriminator network
dis_optimizer = Adam(lr=0.0002, beta_1=0.5, beta_2=0.999, epsilon=10e-8)
# Optimizer for the generator network
gen_optimizer = Adam(lr=0.0002, beta_1=0.5, beta_2=0.999, epsilon=10e-8)
# Optimizer for the adversarial network
adversarial_optimizer = Adam(lr=0.0002, beta_1=0.5, beta_2=0.999, epsilon=10e-8)
Use a learning rate equal to 0.0002, a beta_1 value equal to 0.5, a beta_2 value equal to 0.999, and an epsilon value equal to 10e-8 for all optimizers.
- Next, load and compile the generator and the discriminator networks. In Keras, we must compile the networks before we train the networks:
# Build and compile the discriminator network
discriminator = build_discriminator()
discriminator.compile(loss=['binary_crossentropy'], optimizer=dis_optimizer)
# Build and compile the generator network
generator = build_generator1()
generator.compile(loss=['binary_crossentropy'], optimizer=gen_optimizer)
To compile the networks, use binary_crossentropy as the loss function.
- Next, build and compile the adversarial model, as follows:
# Build and compile the adversarial model
discriminator.trainable = False
input_z_noise = Input(shape=(100,))
input_label = Input(shape=(6,))
recons_images = generator([input_z_noise, input_label])
valid = discriminator([recons_images, input_label])
adversarial_model = Model(inputs=[input_z_noise, input_label], outputs=[valid])
adversarial_model.compile(loss=['binary_crossentropy'], optimizer=gen_optimizer)
To compile the adversarial model, use binary_crossentropy as the loss function and gen_optimizer as the optimizer.
- Next, add TensorBoard to store losses, as follows:
tensorboard = TensorBoard(log_dir="logs/{}".format(time.time()))
tensorboard.set_model(generator)
tensorboard.set_model(discriminator)
- Next, load all images using the load_data function, which is defined in the Preparing the data section:
images, age_list = load_data(wiki_dir=wiki_dir, dataset="wiki")
- Next, convert the age numerical value to the age category, as follows:
# Convert age to category
age_cat = age_to_category(age_list)
The definition of the age_to_category function is as follows:
# This method will convert age to respective category
def age_to_category(age_list):
age_list1 = []
for age in age_list:
if 0 < age <= 18:
age_category = 0
elif 18 < age <= 29:
age_category = 1
elif 29 < age <= 39:
age_category = 2
elif 39 < age <= 49:
age_category = 3
elif 49 < age <= 59:
age_category = 4
elif age >= 60:
age_category = 5
age_list1.append(age_category)
return age_list1
The output of age_cat should look as follows:
[1, 2, 4, 2, 3, 4, 2, 5, 5, 1, 3, 2, 1, 1, 2, 1, 2, 2, 1, 5, 4 , .............]
Convert the age categories to one-hot encoded vectors:
# Also, convert the age categories to one-hot encoded vectors
final_age_cat = np.reshape(np.array(age_cat), [len(age_cat), 1])
classes = len(set(age_cat))
y = to_categorical(final_age_cat, num_classes=len(set(age_cat)))
After we convert the age categories to one-hot encoded vectors, the values of y should be as follows:
[[0. 1. 0. 0. 0. 0.]
[0. 0. 1. 0. 0. 0.]
[0. 0. 0. 0. 1. 0.]
...
[0. 0. 0. 1. 0. 0.]
[0. 1. 0. 0. 0. 0.]
[0. 0. 0. 0. 1. 0.]]
The shape of y should be (total_values, 5).
- Next, load all images and create an ndarray containing all images:
# Read all images and create an ndarray
loaded_images = load_images(wiki_dir, images, (image_shape[0], image_shape[1]))
The definition of the load_images function is as follows:
def load_images(data_dir, image_paths, image_shape):
images = None
for i, image_path in enumerate(image_paths):
print()
try:
# Load image
loaded_image = image.load_img(os.path.join(data_dir, image_path), target_size=image_shape)
# Convert PIL image to numpy ndarray
loaded_image = image.img_to_array(loaded_image)
# Add another dimension (Add batch dimension)
loaded_image = np.expand_dims(loaded_image, axis=0)
# Concatenate all images into one tensor
if images is None:
images = loaded_image
else:
images = np.concatenate([images, loaded_image], axis=0)
except Exception as e:
print("Error:", i, e)
return images
The values inside loaded_images should look like this:
[[[[ 97. 122. 178.]
[ 98. 123. 179.]
[ 99. 124. 180.]
...
[ 97. 124. 179.]
[ 96. 123. 178.]
[ 95. 122. 177.]]
...
[[216. 197. 203.]
[217. 198. 204.]
[218. 199. 205.]
...
[ 66. 75. 90.]
[110. 127. 171.]
[ 89. 115. 172.]]]
[[[122. 140. 152.]
[115. 133. 145.]
[ 95. 113. 123.]
...
[ 41. 73. 23.]
[ 38. 77. 22.]
[ 38. 77. 22.]]
[[ 53. 80. 63.]
[ 47. 74. 57.]
[ 45. 72. 55.]
...
[ 34. 66...
- Next, create a for loop, which should run for the number of times specified by the number of epochs, as follows:
for epoch in range(epochs):
print("Epoch:{}".format(epoch))
gen_losses = []
dis_losses = []
number_of_batches = int(len(loaded_images) / batch_size)
print("Number of batches:", number_of_batches)
- Next, create another loop inside the epochs loop and make it run for the number of times that is specified by num_batches, as follows:
for index in range(number_of_batches):
print("Batch:{}".format(index + 1))
Our entire code for the training of the discriminator networks and the adversarial network will be inside this loop.
- Next, sample a batch of images from the real dataset and a batch of one-hot encoded age vectors:
images_batch = loaded_images[index * batch_size:(index + 1) * batch_size]
images_batch = images_batch / 127.5 - 1.0
images_batch = images_batch.astype(np.float32)
y_batch = y[index * batch_size:(index + 1) * batch_size]
The shape of image_batch should be (batch_size, 64, 64, 3) and the shape of y_batch should be (batch_size, 6).
- Next, sample a batch of noise vectors from a Gaussian distribution, as follows:
z_noise = np.random.normal(0, 1, size=(batch_size, z_shape))
- Next, generate fake images using the generator network. Keep in mind that we haven't trained the generator network yet:
initial_recon_images = generator.predict_on_batch([z_noise, y_batch])
The generator network takes two inputs, z_noise and y_batch, which we created in steps 11 and 12.
- Now, train the discriminator network on real images as well as on fake images:
d_loss_real = discriminator.train_on_batch([images_batch, y_batch], real_labels)
d_loss_fake = discriminator.train_on_batch([initial_recon_images, y_batch], fake_labels)
This code should train the discriminator network on one batch of images. In each step, the discriminator will be trained on a batch of samples.
- Next, train the adversarial network. By freezing the discriminator network, we will train only the generator network:
# Again sample a batch of noise vectors from a Gaussian(normal) distribution
z_noise2 = np.random.normal(0, 1, size=(batch_size, z_shape))
# Samples a batch of random age values
random_labels = np.random.randint(0, 6, batch_size).reshape(-1, 1)
# Convert the random age values to one-hot encoders
random_labels = to_categorical(random_labels, 6)
# Train the generator network
g_loss = adversarial_model.train_on_batch([z_noise2, sampled_labels], [1] * batch_size)
The preceding code will train the generator network on one batch of inputs. The inputs to the adversarial model are z_noise2 and random_labels.
- Next, calculate and print the losses:
d_loss = 0.5 * np.add(d_loss_real, d_loss_fake)
print("d_loss:{}".format(d_loss))
print("g_loss:{}".format(g_loss))
# Add losses to their respective lists
gen_losses.append(g_loss)
dis_losses.append(d_loss)
- Next, write the losses to TensorBoard for visualization:
write_log(tensorboard, 'g_loss', np.mean(gen_losses), epoch)
write_log(tensorboard, 'd_loss', np.mean(dis_losses), epoch)
- Sample and save the images after every 10 epochs, as follows:
if epoch % 10 == 0:
images_batch = loaded_images[0:batch_size]
images_batch = images_batch / 127.5 - 1.0
images_batch = images_batch.astype(np.float32)
y_batch = y[0:batch_size]
z_noise = np.random.normal(0, 1, size=(batch_size, z_shape))
gen_images = generator.predict_on_batch([z_noise, y_batch])
for i, img in enumerate(gen_images[:5]):
save_rgb_img(img, path="results/img_{}_{}.png".format(epoch, i))
Put the preceding code block inside the epochs loop. After every 10 epochs, it will generate a batch of fake images and save them to the results directory. Here, save_rgb_img() is a utility function, defined as follows:
def save_rgb_img(img, path):
"""
Save a rgb image
"""
fig = plt.figure()
ax = fig.add_subplot(1, 1, 1)
ax.imshow(img)
ax.axis("off")
ax.set_title("Image")
plt.savefig(path)
plt.close()
- Finally, save both models by adding the following lines:
# Save weights only
generator.save_weights("generator.h5")
discriminator.save_weights("discriminator.h5")
# Save architecture and weights both
generator.save("generator.h5)
discriminator.save("discriminator.h5")
If you have successfully executed the code given in this section, then you have successfully trained both the generator and the discriminator network. After this step, the generator network will start generating blurred face images. In the next section, we will train the encoder model for the initial latent vector approximation.