Let's use one more callback. This time, we will build a custom callback that computes Receiver Operating Characteristic Area Under the Curve (ROC AUC) at the end of every epoch, on both training and testing sets.  

Creating a custom callback in Keras is actually really simple. All we need to do is create a class, inherent Callback, and override the method we need. Since we want to calculate the ROC AUC score at the end of each epoch, we will override on _epoch_end:

from keras.callbacks import Callback

class RocAUCScore(Callback):
    def __init__(self, training_data, validation_data):
        self.x = training_data[0]
        self.y = training_data[1]
        self.x_val = validation_data[0]
        self.y_val = validation_data[1]
        super(RocAUCScore, self).__init__()

    def on_epoch_end(self, epoch, logs={}):
        y_pred = self.model.predict(self.x)
        roc = roc_auc_score(self.y, y_pred)
        y_pred_val = self.model.predict(self.x_val)
        roc_val = roc_auc_score(self.y_val, y_pred_val)
        print('
  *** ROC AUC Score: %s - roc-auc_val: %s ***' % 
          (str(roc), str(roc_val)))
        return

Now that we've created our new custom callback, we can just add it to our callback creator function, as shown in the following code:

def create_callbacks(data):
   tensorboard_callback = TensorBoard(log_dir=os.path.join(os.getcwd(),  
     "tb_log", "5h_adam_20epochs"), histogram_freq=1, batch_size=32, 
        write_graph=True, write_grads=False)
   roc_auc_callback = RocAUCScore(training_data=(data["train_X"], 
     data["train_y"]), validation_data=(data["val_X"], data["val_y"]))
  checkpoint_callback = ModelCheckpoint(filepath="./model-weights.
    {epoch:02d}-{val_acc:.6f}.hdf5", monitor='val_acc',verbose=1, 
    save_best_only=True)
    return [tensorboard_callback, roc_auc_callback, checkpoint_callback]

That's all there is to it! You can implement any other metric you'd like in the same way.