- Scala Data Analysis Cookbook
- Table of Contents
- Scala Data Analysis Cookbook
- Credits
- About the Author
- About the Reviewers
- www.PacktPub.com
- Preface
- 1. Getting Started with Breeze
- Introduction
- Getting Breeze – the linear algebra library
- Working with vectors
- Getting ready
- How to do it...
- Creating vectors
- Constructing a vector from values
- Creating a vector out of a function
- Creating a vector of linearly spaced values
- Creating a vector with values in a specific range
- Creating an entire vector with a single value
- Slicing a sub-vector from a bigger vector
- Creating a Breeze Vector from a Scala Vector
- Vector arithmetic
- Scalar operations
- Calculating the dot product of two vectors
- Creating a new vector by adding two vectors together
- Appending vectors and converting a vector of one type to another
- Concatenating two vectors
- Standard deviation
- Find the largest value in a vector
- Finding the sum, square root and log of all the values in the vector
- Working with matrices
- Vectors and matrices with randomly distributed values
- How it works...
- Creating vectors with uniformly distributed random values
- Creating vectors with normally distributed random values
- Creating vectors with random values that have a Poisson distribution
- Creating a matrix with uniformly random values
- Creating a matrix with normally distributed random values
- Creating a matrix with random values that has a Poisson distribution
- How it works...
- Reading and writing CSV files
- 2. Getting Started with Apache Spark DataFrames
- Introduction
- Getting Apache Spark
- Creating a DataFrame from CSV
- Manipulating DataFrames
- Creating a DataFrame from Scala case classes
- 3. Loading and Preparing Data – DataFrame
- 4. Data Visualization
- 5. Learning from Data
- Introduction
- Supervised and unsupervised learning
- Gradient descent
- Predicting continuous values using linear regression
- Binary classification using LogisticRegression and SVM
- Binary classification using LogisticRegression with Pipeline API
- How to do it...
- Importing and splitting data as test and training sets
- Construct the participants of the Pipeline
- Preparing a pipeline and training a model
- Predicting against test data
- Evaluating a model without cross-validation
- Constructing parameters for cross-validation
- Constructing cross-validator and fit the best model
- Evaluating the model with cross-validation
- How to do it...
- Clustering using K-means
- Feature reduction using principal component analysis
- How to do it...
- Dimensionality reduction of data for supervised learning
- Mean-normalizing the training data
- Extracting the principal components
- Preparing the labeled data
- Preparing the test data
- Classify and evaluate the metrics
- Dimensionality reduction of data for unsupervised learning
- Mean-normalizing the training data
- Extracting the principal components
- Arriving at the number of components
- Evaluating the metrics
- How to do it...
- 6. Scaling Up
- 7. Going Further
- Index
In this chapter, we will cover the following recipes:
- Using Spark Streaming to subscribe to a Twitter stream
- Using Spark as an ETL tool (pulling data from ElasticSearch and publishing it to Kafka)
- Using StreamingLogisticRegression to classify a Twitter stream using Kafka as a training stream
- Using GraphX to analyze Twitter data
- Watching other Scala libraries of interest
So far, the entire book has concentrated a little around Breeze and a lot around Spark, specifically DataFrames and machine learning. However, there are a whole lot of other libraries, both in Java and Scala that could be leveraged while analyzing data from Scala. This chapter goes a little more into Spark's other components, streaming and GraphX. Note that each recipe in this chapter feeds into the next recipe.
Note
All the code related to this chapter can be downloaded from https://github.com/arunma/ScalaDataAnalysisCookbook/tree/master/chapter7-goingfurther.
-
No Comment
..................Content has been hidden....................
You can't read the all page of ebook, please click here login for view all page.