Deep learning models are often said to be not interpretable. But there are different techniques that are being explored to interpret what happens inside these models. For images, the features learned by convents are interpretable. We will explore two popular techniques to understand convents.