In the last section, we completed our model estimation. Now, it is the time for us to evaluate these estimated models to check whether they fit the city's criterions so that we can either move to the results explanation or go back to some previous stages to refine our predictive models.

To perform our model evaluation, in this section, we will mainly use root mean square error (RMSE) to assess our models for both the regression and time series models. While other measures, such as MSE, can also be used to assess models, as an exercise, we will focus on RMSE as the processes of using other measures are similar.

When working on this real-life project, as mentioned in the Methods of service forecasting section of this chapter, we also used decision tree and random forest models, for which we should use a confusion matrix and error ratios to evaluate. Here, we will not discuss these model evaluation methods as they are used a few times in the previous chapters, such as in Chapter 4, Fraud Detection on Spark.

Similarly to model estimation, to calculate RMSE, we need to use MLlib for regression modeling to be implemented with Zeppeline notebooks on Spark. For time series modeling, we will use R notebooks to be implemented in the Databricks environment of Spark.

val valuesAndPreds = test.map { point =>
  val prediction = new_model.predict(point.features)
  val r = (point.label, prediction)
  r
}
val residuals = valuesAndPreds.map {case (v, p) => math.pow((v - p), 2)}
val MSE = residuals.mean();
val RMSE = math.pow(MSE, 0.5)

accuracy(f, x, test=NULL, d=NULL, D=NULL)