"I really like streaming services. It's a great way for people to find your music"
- Kygo
In this chapter, we will learn about Spark Streaming and find out how we can take advantage of it to process streams of data using the Spark API. Moreover, in this chapter, we will learn various ways of processing real-time streams of data using a practical example to consume and process tweets from Twitter. In a nutshell, the following topics will be covered throughout this chapter:
- A brief introduction to streaming
- Spark Streaming
- Discretized streams
- Stateful/stateless transformations
- Checkpointing
- Interoperability with streaming platforms (Apache Kafka)
- Structured streaming