In ILSVRC 2015, a novel CNN architecture with skip connections and batch normalization was introduced  by Kaiming He and their co-authors from Microsoft Research Asia called the Residual Neural Network (ResNet). With this, they were able to train a neural network with 152 layers (eight times deeper than the VGG network) while still having lower complexity than the VGG network. It achieves a top-5 error rate of 3.57%, which beats human-level performance on this dataset. 

The main idea in this architecture is as follows. Instead of hoping that a set of stacked layers would directly fit a desired underlying mapping, H(x), they tried to fit a residual mapping. More formally, they let the stacked set of layers learn the residual R(x) = H(x) - x, with the true mapping later being obtained by a skip connection. The input is then added to the learned residual, R(x) + x. 

Also, batch normalization is applied right after each convolution and before activation:

Here is the full ResNet architecture compared to VGG-19. The dotted skip connections show an increase in dimensions; hence, for the addition to be valid, no padding is done. Also, increases in dimensions are indicated by changes in color:

All the variants of CNN models that we have talked about so far are available as pretrained models in Keras and TensorFlow. We will be using them heavily in our transfer learning applications. Here is the Keras code snippet used to load various VGG models. More can be found here https://keras.io/applications/: 

from keras.applications.vgg16 import VGG16
model = VGG16()
print(model.summary())