Categorical cross-entropy is a great cost function, but it doesn't actually tell us much about the quality of predictions we can expect from our network. Unfortunately, binary classifications metrics like ROC AUC don't help us much either, as we move beyond binary classification AUC isn't really defined.

Given the lack of a better metric, I'll be using accuracy as a human understandable training metric. Luckily, in this case, my dataset is balanced. Accuracy is defined, as you'd expect, at the number of times the true value matches the predicted value, divided by the total dataset size.

After training is complete, I'll be using scikit-learn's classification report to show us the precision and recall for each class individually. If you like, you could also use a confusion matrix for this.