When trying to train a classifier, we will often end up with multiple variables for which we don't know a good setting. Viewing values used by solutions for similar problems is a good starting point. However, we are often left with an array of possible values that we need to test. To make things more complicated, we often have several such parameters, resulting in numerous combinations that we may need to test.

For such situations, we suggest keeping the parameters of interest as values that can be passed into the trainer. Then, a wrapper script can pass in various combinations of the parameters, along with a unique output log subdirectory that is possibly tagged with a descriptive name.

This will allow an easy comparison of results and intermediate values across multiple tests. The following figure shows four runs' losses plotted together. We can easily see the underperforming and overperforming pairs: