This chapter started with an architectural overview on GraphFrames. We saw how GraphFrames can make use of the Catalyst and Tungsten optimizers by running on top of DataFrames.

Additional optimizations on top of these were explained. Finally, we showed, by example, how Scala-based code can be used to call GraphFrames algorithms in Apache Spark. Scala has been used because it requires less code to develop the examples, which saves time; a Scala-based shell can be used and the code can be compiled into Spark applications.

The configuration and code examples from this chapter are also available for download with the book. If you want to learn more on GraphFrames please refer to this link https://databricks.com/blog/2016/03/03/introducing-graphframes.html.

Now let's have a look how Apache Spark can be used in the Cloud in conjunction with Jupyter in the next chapter. This is a very convenient user interface from which to do data science.