- Big Data Analytics
- Table of Contents
- Big Data Analytics
- Credits
- About the Author
- Acknowledgement
- About the Reviewers
- www.PacktPub.com
- Preface
- 1. Big Data Analytics at a 10,000-Foot View
- 2. Getting Started with Apache Hadoop and Apache Spark
- 3. Deep Dive into Apache Spark
- Starting Spark daemons
- Learning Spark core concepts
- Lifecycle of Spark program
- Spark applications
- Persistence and caching
- Spark resource managers – Standalone, YARN, and Mesos
- Summary
- 4. Big Data Analytics with Spark SQL, DataFrames, and Datasets
- History of Spark SQL
- Architecture of Spark SQL
- Introducing SQL, Datasources, DataFrame, and Dataset APIs
- Evolution of DataFrames and Datasets
- Why Datasets and DataFrames?
- When to use RDDs, Datasets, and DataFrames?
- Analytics with DataFrames
- Analytics with the Dataset API
- Data Sources API
- Spark SQL as a distributed SQL engine
- Hive on Spark
- Summary
- 5. Real-Time Analytics with Spark Streaming and Structured Streaming
- 6. Notebooks and Dataflows with Spark and Hadoop
- 7. Machine Learning with Spark and Hadoop
- 8. Building Recommendation Systems with Spark and Mahout
- 9. Graph Analytics with GraphX
- 10. Interactive Analytics with SparkR
- Index
The following table describes the scenarios in which RDDs, Datasets, or DataFrames are to be used:
|
Scenario |
What to use? |
|---|---|
|
Use of the Python programming language |
RDDs or DataFrames |
|
Use of the R programming language |
DataFrames |
|
Use of the Java or Scala programming languages |
RDDs, Datasets, or DataFrames |
|
Unstructured data such as images and videos |
RDDs |
|
Use of low level transformations, actions, and controls data flow programmatically |
RDDs |
|
Use of high-level domain-specific APIs |
Datasets and DataFrames |
|
Use of functional programming constructs to process data |
RDDs |
|
Datasets and DataFrames | |
|
Imposing structure is not needed and low-level optimizations are not needed |
RDDs |
|
High compile time safety and rich optimizations |
Datasets |
|
DataFrames | |
|
Unification is needed across Spark libraries |
Datasets or DataFrames |
-
No Comment