A great feature of Pachyderm—particularly for enterprise use cases—is the ability to version both your models and your predictions. Say you are predicting the chance a customer will repay a loan, and you see a batch of strange predictions. As part of troubleshooting why the model has made these decisions, if you are training multiple models across a large team, trawling through email chains and commit histories would be a bad idea!

So, work backward from the inference to the model, and simply run the following command:

#pachctl list job

You can then take the relevant commit hash and feed it to the following command, observing the details of the output:

#pachctl inspect job 9339d8d712d945d58322a5ac649d9239
...
Input:
{
  "pfs": {
    "name": "data",
    "repo": "data",
    "branch": "master",
    "commit": "b22db05d23324ede839718bec5ff219c",
    "glob": "/"
  }
}
...

#pachctl inspect commit data@b22db05d23324ede839718bec5ff219c
Commit: data@b22db05d23324ede839718bec5ff219c
Original Branch: master
Started: 11 minutes ago
Finished: 11 minutes ago
Size: 202.8MiB

You can see the exact commit of the model that was used to produce this prediction, the prediction's provenance, and in turn, the data that was used to train the model:

#pachctl list file data@adb293f8a4604ed7b081c1ff030c0480
COMMIT NAME TYPE COMMITTED SIZE
b22db05d23324ede839718bec5ff219c /data dir 11 minutes ago 202.8MiB