In this section, we will look at how to develop a cancer diagnosis pipeline with Spark ML. A real dataset will be used to predict the probability of breast cancer. To be more specific, Wisconsin Breast Cancer Dataset will be used.
Breast cancer prediction using logistic regression of Spark ML
by Md. Rezaul Karim
Scala and Spark for Big Data Analytics
- Preface
- Introduction to Scala
- Object-Oriented Scala
- Functional Programming Concepts
- Collection APIs
- Tackle Big Data – Spark Comes to the Party
- Start Working with Spark – REPL and RDDs
- Special RDD Operations
- Introduce a Little Structure - Spark SQL
- Stream Me Up, Scotty - Spark Streaming
- Everything is Connected - GraphX
- Learning Machine Learning - Spark MLlib and Spark ML
- Introduction to machine learning
- Spark machine learning APIs
- Feature extraction and transformation
- Creating a simple pipeline
- Unsupervised machine learning
- Binary and multiclass classification
- Summary
- My Name is Bayes, Naive Bayes
- Time to Put Some Order - Cluster Your Data with Spark MLlib
- Text Analytics Using Spark ML
- Spark Tuning
- Time to Go to ClusterLand - Deploying Spark on a Cluster
- Spark architecture in a cluster
- Deploying the Spark application on a cluster
- Submitting Spark jobs
- Hadoop YARN
- Configuring a single-node YARN cluster
- Step 1: Downloading Apache Hadoop
- Step 2: Setting the JAVA_HOME
- Step 3: Creating users and groups
- Step 4: Creating data and log directories
- Step 5: Configuring core-site.xml
- Step 6: Configuring hdfs-site.xml
- Step 7: Configuring mapred-site.xml
- Step 8: Configuring yarn-site.xml
- Step 9: Setting Java heap space
- Step 10: Formatting HDFS
- Step 11: Starting the HDFS
- Step 12: Starting YARN
- Step 13: Verifying on the web UI
- Submitting Spark jobs on YARN cluster
- Advance job submissions in a YARN cluster
- Configuring a single-node YARN cluster
- Apache Mesos
- Deploying on AWS
- Summary
- Testing and Debugging Spark
- PySpark and SparkR
Breast cancer prediction using logistic regression of Spark ML
-
No Comment
..................Content has been hidden....................
You can't read the all page of ebook, please click here login for view all page.