In addition to the split of data, distribution of data has huge impact on the performance of a neural network. Most issues in applied deep learning come from the mismatch of the dev and test set data distribution. We need to bear in mind that the dev and test data should be coming from a similar distribution. For example, we will have a distribution mismatch if we collect and split person detection data in a way that training images of people are collected from web pages while test set images are collected using mobile phones. The problem here is that while training a model we finetune the parameters and architecture of a network based on its performance on the dev data, and if the dev data is similar to training data and different from test data then there is high bias in dev data towards the training set. Good evaluation results on the dev set do not necessarily mean that the model will generalize well. In this case, testing on a set with an entirely different distribution may evaluate to bad results. This is a waste of time and effort. The solution to this is to initially merge the dev and test sets, randomly shuffle them and finally split the shuffled data into dev and test sets again. This helps make faster progress in training machine learning algorithms successfully to your final application.