Normally, in order to take advantage of Spark, data is stored in a Hadoop Distributed File System (HDFS), which is a distributed filesystem designed to store large volumes of data, and computation occurs over multiple nodes on clusters. For demonstration purposes, we are keeping the data on a local machine and running Spark locally. It is no different from running it on a distributed computing cluster.
Learning on massive click logs with Spark
by Yuxi Liu
Python Machine Learning By Example - Second Edition
- Title Page
- Copyright and Credits
- About Packt
- Dedication
- Foreword
- Contributors
- Preface
- Section 1: Fundamentals of Machine Learning
- Getting Started with Machine Learning and Python
- Section 2: Practical Python Machine Learning By Example
- Exploring the 20 Newsgroups Dataset with Text Analysis Techniques
- Mining the 20 Newsgroups Dataset with Clustering and Topic Modeling Algorithms
- Detecting Spam Email with Naive Bayes
- Classifying Newsgroup Topics with Support Vector Machines
- Finding separating boundary with support vector machines
- Classifying newsgroup topics with SVMs
- More example – fetal state classification on cardiotocography
- A further example – breast cancer classification using SVM with TensorFlow
- Summary
- Exercise
- Predicting Online Ad Click-Through with Tree-Based Algorithms
- Brief overview of advertising click-through prediction
- Getting started with two types of data – numerical and categorical
- Exploring decision tree from root to leaves
- Implementing a decision tree from scratch
- Predicting ad click-through with decision tree
- Ensembling decision trees – random forest
- Summary
- Exercise
- Predicting Online Ad Click-Through with Logistic Regression
- Converting categorical features to numerical – one-hot encoding and ordinal encoding
- Classifying data with logistic regression
- Training a logistic regression model
- Training on large datasets with online learning
- Handling multiclass classification
- Implementing logistic regression using TensorFlow
- Feature selection using random forest
- Summary
- Exercises
- Scaling Up Prediction to Terabyte Click Logs
- Stock Price Prediction with Regression Algorithms
- Brief overview of the stock market and stock prices
- What is regression?
- Mining stock price data
- Estimating with linear regression
- Estimating with decision tree regression
- Estimating with support vector regression
- Estimating with neural networks
- Evaluating regression performance
- Predicting stock price with four regression algorithms
- Summary
- Exercise
- Section 3: Python Machine Learning Best Practices
- Machine Learning Best Practices
- Machine learning solution workflow
- Best practices in the data preparation stage
- Best practice 1 – completely understanding the project goal
- Best practice 2 – collecting all fields that are relevant
- Best practice 3 – maintaining the consistency of field values
- Best practice 4 – dealing with missing data
- Best practice 5 – storing large-scale data
- Best practices in the training sets generation stage
- Best practice 6 – identifying categorical features with numerical values
- Best practice 7 – deciding on whether or not to encode categorical features
- Best practice 8 – deciding on whether or not to select features, and if so, how to do so
- Best practice 9 – deciding on whether or not to reduce dimensionality, and if so, how to do so
- Best practice 10 – deciding on whether or not to rescale features
- Best practice 11 – performing feature engineering with domain expertise
- Best practice 12 – performing feature engineering without domain expertise
- Best practice 13 – documenting how each feature is generated
- Best practice 14 – extracting features from text data
- Best practices in the model training, evaluation, and selection stage
- Best practices in the deployment and monitoring stage
- Summary
- Exercises
- Other Books You May Enjoy
Learning on massive click logs with Spark
-
No Comment
..................Content has been hidden....................
You can't read the all page of ebook, please click here login for view all page.