Frank Kane's Taming Big Data with Apache Spark and Python
by Frank Kane
Frank Kane's Taming Big Data with Apache Spark and Python
- Preface
- Getting Started with Spark
- Spark Basics and Spark Examples
- What is Spark?
- The Resilient Distributed Dataset (RDD)
- Ratings histogram walk-through
- Key/value RDDs and the average friends by age example
- Running the average friends by age example
- Filtering RDDs and the minimum temperature by location example
- Running the minimum temperature example and modifying it for maximums
- Running the maximum temperature by location example
- Counting word occurrences using flatmap()
- Improving the word-count script with regular expressions
- Sorting the word count results
- Find the total amount spent by customer
- Check your results and sort them by the total amount spent
- Check your sorted implementation and results against mine
- Summary
- Advanced Examples of Spark Programs
- Finding the most popular movie
- Using broadcast variables to display movie names instead of ID numbers
- Finding the most popular superhero in a social graph
- Running the script - discover who the most popular superhero is
- Superhero degrees of separation - introducing the breadth-first search algorithm
- Accumulators and implementing BFS in Spark
- Superhero degrees of separation - review the code and run it
- Item-based collaborative filtering in Spark, cache(), and persist()
- Running the similar-movies script using Spark's cluster manager
- Improving the quality of the similar movies example
- Summary
- Running Spark on a Cluster
- Introducing Elastic MapReduce
- Setting up our Amazon Web Services / Elastic MapReduce account and PuTTY
- Partitioning
- Creating similar movies from one million ratings - part 1
- Creating similar movies from one million ratings - part 2
- Creating similar movies from one million ratings – part 3
- Troubleshooting Spark on a cluster
- More troubleshooting and managing dependencies
- Summary
- SparkSQL, DataFrames, and DataSets
- Other Spark Technologies and Libraries
- Where to Go From Here? – Learning More About Spark and Data Science
-
No Comment
..................Content has been hidden....................
You can't read the all page of ebook, please click here login for view all page.