Basic naive Bayes classifier baseline
by Krishna Choppella, Dr. Uday Kamath, Boštjan Kaluža, Jennifer L. Reese, Richard M
Machine Learning: End-to-End guide for Java developers
- Machine Learning: End-to-End guide for Java developers
- Table of Contents
- Machine Learning: End-to-End guide for Java developers
- Credits
- Preface
- 1. Module 1
- 1. Getting Started with Data Science
- Problems solved using data science
- Understanding the data science problem - solving approach
- Acquiring data for an application
- The importance and process of cleaning data
- Visualizing data to enhance understanding
- The use of statistical methods in data science
- Machine learning applied to data science
- Using neural networks in data science
- Deep learning approaches
- Performing text analysis
- Visual and audio analysis
- Improving application performance using parallel techniques
- Assembling the pieces
- Summary
- 2. Data Acquisition
- 3. Data Cleaning
- 4. Data Visualization
- 5. Statistical Data Analysis Techniques
- 6. Machine Learning
- 7. Neural Networks
- 8. Deep Learning
- 9. Text Analysis
- 10. Visual and Audio Analysis
- 11. Mathematical and Parallel Techniques for Data Analysis
- 12. Bringing It All Together
- 1. Getting Started with Data Science
- 2. Module 2
- 1. Applied Machine Learning Quick Start
- 2. Java Libraries and Platforms for Machine Learning
- 3. Basic Algorithms – Classification, Regression, and Clustering
- 4. Customer Relationship Prediction with Ensembles
- 5. Affinity Analysis
- 6. Recommendation Engine with Apache Mahout
- 7. Fraud and Anomaly Detection
- 8. Image Recognition with Deeplearning4j
- 9. Activity Recognition with Mobile Phone Sensors
- 10. Text Mining with Mallet – Topic Modeling and Spam Detection
- 11. What is Next?
- A. References
- 3. Module 3
- 1. Machine Learning Review
- Machine learning – history and definition
- What is not machine learning?
- Machine learning – concepts and terminology
- Machine learning – types and subtypes
- Datasets used in machine learning
- Machine learning applications
- Practical issues in machine learning
- Machine learning – roles and process
- Machine learning – tools and datasets
- Summary
- 2. Practical Approach to Real-World Supervised Learning
- Formal description and notation
- Data transformation and preprocessing
- Feature relevance analysis and dimensionality reduction
- Model building
- Model assessment, evaluation, and comparisons
- Case Study – Horse Colic Classification
- Summary
- References
- 3. Unsupervised Machine Learning Techniques
- Issues in common with supervised learning
- Issues specific to unsupervised learning
- Feature analysis and dimensionality reduction
- Clustering
- Outlier or anomaly detection
- Real-world case study
- Summary
- References
- 4. Semi-Supervised and Active Learning
- Semi-supervised learning
- Active learning
- Case study in active learning
- Summary
- References
- 5. Real-Time Stream Machine Learning
- Assumptions and mathematical notations
- Basic stream processing and computational techniques
- Concept drift and drift detection
- Incremental supervised learning
- Incremental unsupervised learning using clustering
- Modeling techniques
- Partition based
- Hierarchical based and micro clustering
- Density based
- Grid based
- Validation and evaluation techniques
- Key issues in stream cluster evaluation
- Evaluation measures
- Cluster Mapping Measures (CMM)
- V-Measure
- Other external measures
- Unsupervised learning using outlier detection
- Case study in stream learning
- Summary
- References
- 6. Probabilistic Graph Modeling
- Probability revisited
- Graph concepts
- Bayesian networks
- Representation
- Inference
- Learning
- Learning parameters
- Learning structures
- Measures to evaluate structures
- Methods for learning structures
- Constraint-based techniques
- Search and score-based techniques
- 7. Deep Learning
- Multi-layer feed-forward neural network
- Inputs, neurons, activation function, and mathematical notation
- Multi-layered neural network
- Structure and mathematical notations
- Activation functions in NN
- Training neural network
- Empirical risk minimization
- Parameter initialization
- Loss function
- Gradients
- Feed forward and backpropagation
- How does it work?
- Regularization
- L2 regularization
- L1 regularization
- Limitations of neural networks
- Deep learning
- Building blocks for deep learning
- Rectified linear activation function
- Restricted Boltzmann Machines
- Autoencoders
- Unsupervised pre-training and supervised fine-tuning
- Deep feed-forward NN
- Deep Autoencoders
- Deep Belief Networks
- Deep learning with dropouts
- Sparse coding
- Convolutional Neural Network
- CNN Layers
- Recurrent Neural Networks
- Building blocks for deep learning
- Case study
- Summary
- References
- 8. Text Mining and Natural Language Processing
- NLP, subfields, and tasks
- Text categorization
- Part-of-speech tagging (POS tagging)
- Text clustering
- Information extraction and named entity recognition
- Sentiment analysis and opinion mining
- Coreference resolution
- Word sense disambiguation
- Machine translation
- Semantic reasoning and inferencing
- Text summarization
- Automating question and answers
- Issues with mining unstructured data
- Text processing components and transformations
- Topics in text mining
- Tools and usage
- Summary
- References
- NLP, subfields, and tasks
- 9. Big Data Machine Learning – The Final Frontier
- What are the characteristics of Big Data?
- Big Data Machine Learning
- Batch Big Data Machine Learning
- Case study
- Business problem
- Machine Learning mapping
- Data collection
- Data sampling and transformation
- Spark MLlib as Big Data Machine Learning platform
- A. Linear Algebra
- B. Probability
- D. Bibliography
- Index
- Empirical risk minimization
- Multi-layer feed-forward neural network
- Modeling techniques
- 1. Machine Learning Review
As per the rules of the challenge, the participants had to outperform the basic naive Bayes classifier to qualify for prizes, which makes an assumption that features are independent (refer to Chapter 1, Applied Machine Learning Quick Start).
The KDD Cup organizers run the vanilla naive Bayes classifier, without any feature selection or hyperparameter adjustments. For the large dataset, the overall scores of the naive Bayes on the test set were as follows:
- Churn problem: AUC = 0.6468
- Appetency problem: AUC = 0.6453
- Upselling problem: AUC=0.7211
Note that the baseline results are reported for large dataset only. Moreover, while both training and test datasets are provided at the KDD Cup site, the actual true labels for the test set are not provided. Therefore, when we process the data with our models, there is no way to know how well the models will perform on the test set. What we will do is use only the training data and evaluate our models with cross validation. The results will not be directly comparable, but, nevertheless, we have an idea for what a reasonable magnitude of the AUC score is.
At the KDD Cup web page (http://kdd.org/kdd-cup/view/kdd-cup-2009/Data), you should see a page that looks like the following screenshot. First, under the Small version (230 var.) header, download orange_small_train.data.zip. Next, download the three sets of true labels associated with this training data. The following files are found under the Real binary targets (small) header:
orange_small_train_appentency.labelsorange_small_train_churn.labelsorange_small_train_upselling.labels
Save and unzip all the files marked in the red boxes, as shown in the following screenshot:
In the following sections, we will first load the data into Weka and apply basic modeling with the naive Bayes to obtain our own baseline AUC scores. Later, we will look into more advanced modeling techniques and tricks.
We will load the data to Weka directly from the .cvs format. For this purpose, we will write a function that accepts the path to the data file and the true labels file. The function will load and merge both datasets and remove empty attributes:
public static Instances loadData(String pathData, String pathLabeles) throws Exception {First, we load the data using the CSVLoader() class. Additionally, we specify the tab as a field separator and force the last 40 attributes to be parsed as nominal:
// Load data
CSVLoader loader = new CSVLoader();
loader.setFieldSeparator(" ");
loader.setNominalAttributes("191-last");
loader.setSource(new File(pathData));
Instances data = loader.getDataSet();Note
The CSVLoader class accepts many additional parameters specifying column separator, string enclosures, whether a header row is present or not, and so on. Complete documentation is available here:
http://weka.sourceforge.net/doc.dev/weka/core/converters/CSVLoader.html
Next, some of the attributes do not contain a single value and Weka automatically recognizes them as the String attributes. We actually do not need them, so we can safely remove them using the RemoveType filter. Additionally, we specify the –T parameters, which means remove attribute of specific type and the attribute type that we want to remove:
// remove empty attributes identified as String attribute
RemoveType removeString = new RemoveType();
removeString.setOptions(new String[]{"-T", "string"});
removeString.setInputFormat(data);
Instances filteredData = Filter.useFilter(data, removeString);Alternatively, we could use the void deleteStringAttributes() method implemented within the Instances class, which has the same effect, for example, data.removeStringAttributes().
Now, we will load and assign class labels to the data. We will again utilize CVSLoader, where we specify that the file does not have any header line, that is, setNoHeaderRowPresent(true):
// Load labeles
loader = new CSVLoader();
loader.setFieldSeparator(" ");
loader.setNoHeaderRowPresent(true);
loader.setNominalAttributes("first-last");
loader.setSource(new File(pathLabeles));
Instances labels = loader.getDataSet();Once we have loaded both files, we can merge them together by calling the Instances.mergeInstances (Instances, Instances) static method. The method returns a new dataset that has all the attributes from the first dataset plus the attributes from the second set. Note that the number of instances in both datasets must be the same:
// Append label as class value Instances labeledData = Instances.mergeInstances(filteredData, labeles);
Finally, we set the last attribute, that is, the label attribute that we have just added, as a target variable and return the resulting dataset:
// set the label attribute as class labeledData.setClassIndex(labeledData.numAttributes() - 1); System.out.println(labeledData.toSummaryString()); return labeledData; }
The function outputs a summary as shown in the following and returns the labeled dataset:
Relation Name: orange_small_train.data-weka.filters.unsupervised.attribute.RemoveType-Tstring_orange_small_train_churn.labels.txt Num Instances: 50000 Num Attributes: 215 Name Type Nom Int Real Missing Unique Dist 1 Var1 Num 0% 1% 0% 49298 / 99% 8 / 0% 18 2 Var2 Num 0% 2% 0% 48759 / 98% 1 / 0% 2 3 Var3 Num 0% 2% 0% 48760 / 98% 104 / 0% 146 4 Var4 Num 0% 3% 0% 48421 / 97% 1 / 0% 4 ...
-
No Comment