Anyone involved in deep learning with images should become familiar with AlexNet. The network was introduced in the landmark paper, ImageNet Classification with Deep Convolutional Neural Networks, by Alex Krizhevsky, Ilya Sutskever, and Geoffrey E. Hinton. The paper can be viewed at http://www.cs.toronto.edu/~fritz/absps/imagenet.pdf.

This network architecture achieved then-record accuracy on the annual ImageNet competition. The architecture is described in their paper, as shown in the following image. We will be using this network architecture in later chapters, but for now, let's browse through the network using TensorBoard:

We will not review line-by-line changes to the existing AlexNet code, but the reader can easily see changes by noting differences between the original model code provided by Google and the revised code that we have included with the book's code repository.

The original AlexNet TensorFlow implementation from Google is available at:

https://github.com/tensorflow/models/blob/master/tutorials/image/alexnet/alexnet_benchmark.py.

The revised AlexNet TensorFlow implementation with TensorBoard instrumentation can be found at:

https://github.com/mlwithtf/mlwithtf/blob/master/chapter_03/alexnet_benchmark.py.

The changes introduced are very similar to those done for our MNIST example.

First, find the location of this code:

    sess = tf.Session(config=config) 
    sess.run(init) 

Then, replace it with the following code:

    sess = tf.Session(config=config) 
    writer = tf.summary.FileWriter("/tmp/alexnet_logs",  
    sess.graph) 
    sess.run(init) 

Finally, you can run the Python file alexnet_benchmark.py and TensorBoard command to visualize the graph:

python alexnet_benchmark.py
tensorboard --logdir /tmp/alexnet_logs

Our focus for this section is just the graph. The following figure shows a section of the Graph Explorer. We have deep dived into convolutional layer 3 of 5 and we are looking at weights and biases for this layer.

Clicking on the weights node on the graph is interesting because we see details such as the shape: {"shape":{"dim":[{"size":3},{"size":3},{"size":192},{"size":384}]}}. We can match many of these details right back to the original paper and the previously referenced diagram! We can also trace details back to the network setup in the code:

    with tf.name_scope('conv3') as scope: 
      kernel = tf.Variable(tf.truncated_normal([3, 3, 192, 384], 
                               dtype=tf.float32, 
                               stddev=1e-1), name='weights') 
      conv = tf.nn.conv2d(pool2, kernel, [1, 1, 1, 1], 
       padding='SAME') 
      biases = tf.Variable(tf.constant(0.0, shape=[384], 
       dtype=tf.float32), 
                         trainable=True, name='biases') 
      bias = tf.nn.bias_add(conv, biases) 
      conv3 = tf.nn.relu(bias, name=scope) 
      parameters += [kernel, biases] 

The details in the Graph Explorer and code are equivalent, but the flow of data is very easily visualized using TensorBoard. It is also easy to collapse repetitive sections and expand sections of interest:

The graph is the most interesting part of this section, but of course, you can also run our revised script and review the training performance, as well as a host of other data we're capturing. You can even capture additional data. Give it a try!