It will be more fulfilling, however, to see our training in action and how we will improve upon what we did earlier.

We will prepare the training dataset and labels as follows:

    tf_train_dataset = tf.placeholder(tf.float32, 
    shape=(batch_size, image_size, image_size,   
    num_channels), 
    name='TRAIN_DATASET')    
    tf_train_labels = tf.placeholder(tf.float32, 
    shape=(batch_size, num_of_classes), 
    name='TRAIN_LABEL') 
    tf_valid_dataset = tf.constant(dataset.valid_dataset,   
    name='VALID_DATASET') 
    tf_test_dataset = tf.constant(dataset.test_dataset,  
    name='TEST_DATASET') 

Then, we will run the trainer, as follows:

    # Training computation. 
    logits = nn_model(tf_train_dataset, weights, biases,  
    True) 
    loss = tf.reduce_mean( 
        tf.nn.softmax_cross_entropy_with_logits(logits, 
         tf_train_labels)) 
    # L2 regularization for the fully connected  
    parameters. 
    regularizers = (tf.nn.l2_loss(weights['fc1']) + 
     tf.nn.l2_loss(biases['fc1']) + 
     tf.nn.l2_loss(weights['fc2']) + 

    tf.nn.l2_loss(biases['fc2'])) 
    # Add the regularization term to the loss. 
    loss += 5e-4 * regularizers 
    tf.summary.scalar("loss", loss) 

This is very similar to what we did in Chapter 2, Your First Classifier. We instantiated the network, passed in an initial set of weights and biases, and defined a loss function using the training labels. Our optimizer is then defined to minimize that loss, as follows:

    optimizer = tf.train.GradientDescentOptimizer
     (learning_rate).minimize(loss)

We will then use the weights and biases to predict labels for the validation and, eventually, the training set:

    train_prediction = tf.nn.softmax(nn_model(tf_train_dataset,  
    weights, biases, TRAIN=False)) 
    valid_prediction = tf.nn.softmax(nn_model(tf_valid_dataset, 
     weights, biases))    test_prediction =  
     tf.nn.softmax(nn_model(tf_test_dataset, 
     weights, biases))

The complete code for training session is as follows:

Finally, we will run the session. We will use the num_steps variable that we set earlier and run through the training data in chunks (batch_size.) We will load small chunks of the training data and associated labels, and run the session as follows:

    batch_data = dataset.train_dataset[offset:(offset + 
     batch_size), :]   
    batch_labels = dataset.train_labels[offset: 
     (offset + 
     batch_size), :]

We will get back predictions on the minibatch, which we will compare against the actual labels to get accuracy on the minibatch.

We will use the following valid_prediction that we declared earlier:

    valid_prediction =   
    tf.nn.softmax(nn_model(tf_valid_dataset, 
     weights, biases))

Then, we will evaluate the validation set predictions against the actual labels we know, as follows:

    accuracy(valid_prediction.eval(), 
    dataset.valid_labels) 

After we've run through all the steps, we will do the same in our test set:

    accuracy(test_prediction.eval(), dataset.test_labels)

As you can see the actual execution of the training, validation, and test was not that different from before. What is different from before is the accuracy. Notice that we've broken out of the 80s into the 90s on test set accuracy: