- Michael Feathers

Using Scala and Spark is a very good combination for learning big data analytics. However, along with the OOP paradigm, we also need to know-how why functional concepts are important for writing Spark applications that eventually analyze your data. As mentioned in the previous chapters, Scala supports two programming paradigms: the Object-Oriented Programming paradigm and the Functional programming concepts. In Chapter 2, Object-Oriented Scala, we explored the OOP paradigm in which we have seen how to represent real-world objects in blueprints (classes) and then instantiate them into objects having real memory representation.

In this chapter, we will focus on the second paradigm (i.e. functional programming). We will see what functional programming is and how Scala supports it, why it matters, and the related advantages of using this concept. More specifically, we will learn several topics, such as why Scala is an arsenal for the data scientist, why it is important to learn the Spark paradigm, pure functions, and higher-order functions (HOFs). A real-life use case using HOF will also be shown in this chapter. Then, we will see how to handle exceptions in the higher-order functions outside collections using the standard library of Scala. Finally, we will learn how functional Scala affects an object's mutability.

In a nutshell, the following topics will be covered in this chapter:

• Introduction to functional programming
• Functional Scala for the data scientists
• Why functional programming and Scala are important for learning Spark?
• Pure functions and higher-order functions
• Using higher-order functions: A real-life use case
• Error handling in functional Scala
• Functional programming and data mutability