Here is how we proceed with the recipe:

1. The first step in tuning hyperparameters is building the model. Build the model in TensorFlow, exactly the way we have been.
2. Add a way to save the model in model_file. In TensorFlow, this can be done using a Saver object. Then save it in the session:

... saver = tf.train.Saver() ... with tf.Session() as sess: ... #Do the training steps ... save_path = saver.save(sess, "/tmp/model.ckpt") print("Model saved in file: %s" % save_path)

3. Next, identify the hyperparameters that you want to tune.
4. Choose possible values for the hyperparameters. Here, you can make a random choice, constant spaced choice, or manual choice. The three are respectively known as random search, grid search, or manual search for optimized hyperparameters. For example, this is for the learning rate:

# Random Choice: generate 5 random values of learning rate 
# lying between 0 and 1 
learning_rate = np.random.rand(5) 
#Grid Search: generate 5 values starting from 0, separated by 
# 0.2 
learning_rate = [i for i in np.arange(0,1,0.2)] 
#Manual Search: give any values you seem plausible manually learning_rate = [0.5, 0.6, 0.32, 0.7, 0.01]

5. We choose the parameters that give the best response to our chosen loss function. So, we can define a maximum value of loss function at the start as best_loss (in the case of accuracy, you will choose the minimum accuracy you desire from your model):

best_loss = 2 
# It can be any number, but it would be better if you keep it same as the loss you achieved from your base model defined in steps 1 and 2

6. Wrap your model in a for loop for the learning rate; any model that gives a better estimate of loss is then saved:

... # Load and preprocess data 
... # Hyperparameters 
Tuning epochs = [50, 60, 70] 
batches = [5, 10, 20] 
rmse_min = 0.04 
for epoch in epochs: 
    for batch in batches: 
        model = get_model() 
        model.compile(loss='mean_squared_error', optimizer='adam') 
        model.fit(X_train, y_train, validation_data=(X_test, y_test),epochs=epoch, batch_size=batch, verbose=1) 
        y_test_pred = model.predict(X_test) 
        rmse = mean_squared_error( y_test, y_test_pred ) 
        if rmse < rmse_min: 
            rmse_min = rmse 
            # serialize model to JSON 
            model_json = model.to_json() 
            with open("model.json", "w") as json_file: 
                json_file.write(model_json) 
                # serialize weights to HDF5  
                model.save_weights("model.hdf5") 
                print("Saved model to disk")