For regression tasks, the most common cost functions are Root Mean Squared Error (RMSE) and Mean Absolute Error (MAE). I'll be using MAE here. It is defined as follows:
Very simply, MAE is the average unsigned error for all examples in the dataset. It's very similar to RMSE; however, we use the absolute value of the difference between y and 
instead of the square root of the average squared error:
You might be wondering how MAE differs from the more familiar RMSE. In cases where the error is evenly distributed across the dataset, RMSE and MAE will be equal. In cases where there are very large outliers in a dataset, RMSE will be much larger than MAE. Your choice of cost function should be appropriate to your use case. In regard to interpretability, MAE is more interpretable than RMSE as it's the actual average error.