But it's not all rainbows and kittens, there are some drawbacks to using a neural network for these really straightforward problems. The most notable drawbacks are:

• As previously noted, neural networks aren't easily interpretable.
• Neural Networks work best when there are many features and a lot of data. Many simple regression problems aren't large enough to really benefit from Neural Networks.
• Much of the time a traditional multiple regression, or a tree model such as Gradient Boosted Trees will outperform a neural network on problems such as this. The more complex, the better the fit for neural networks.