This chapter has shown, with the help of examples, how the Scala-based code can be used to call GraphX algorithms in Apache Spark. Scala has been used, because it requires less code to develop the examples, which saves time. A Scala-based shell can be used, and the code can be compiled into Spark applications. Examples of the application compilation and configuration have been supplied using the SBT tool. The configuration and the code examples from this chapter will also be available for download with the book.

Finally, the Mazerunner example architecture (developed by Kenny Bastani while at Neo) for Neo4j and Apache Spark has been introduced. Why is Mazerunner important? It provides an example of how a graph-based database can be used for graph storage, while Apache Spark is used for graph processing. I am not suggesting that Mazerunner be used in a production scenario at this time. Clearly, more work needs to be done to make this architecture ready for release. However, graph-based storage, when associated with the graph-based processing within a distributed environment, offers the option to interrogate the data using a query language such as Cypher from Neo4j.

I hope that you have found this chapter useful. The next chapter will delve into graph-based storage in more depth. You can now delve into further GraphX coding, try to run the examples provided, and try modifying the code, so that you become familiar with the development process.