In this chapter, we have refocused our efforts on machine learning libraries, especially the MLlib with which we processed data on Spark, and then built models to predict customer churns and develop scores to help the company YST to improve their customer retention.

Specifically, we first selected regression models and decision tree models as per business needs after we prepared Spark computing and loaded in pre-processed data. We then worked on feature extraction with MLlib. Then we estimated the model coefficients with distributed computing. Further, we evaluated these estimated models by using a confusion matrix and false positive ratios as well as RMSE. Then we interpreted our machine learning results. And finally, we deployed our machine learning results with our focus on scoring along with using insights to design interventions.

After this chapter, readers will have gained a better understanding of how Apache Spark, with its machine learning libraries, can be utilized to make our work easier and faster in conducting supervised machine learning, and developing customer retention systems.