Building a machine learning application
by Krishna Choppella, Dr. Uday Kamath, Boštjan Kaluža, Jennifer L. Reese, Richard M
Machine Learning: End-to-End guide for Java developers
- Machine Learning: End-to-End guide for Java developers
- Table of Contents
- Machine Learning: End-to-End guide for Java developers
- Credits
- Preface
- 1. Module 1
- 1. Getting Started with Data Science
- Problems solved using data science
- Understanding the data science problem - solving approach
- Acquiring data for an application
- The importance and process of cleaning data
- Visualizing data to enhance understanding
- The use of statistical methods in data science
- Machine learning applied to data science
- Using neural networks in data science
- Deep learning approaches
- Performing text analysis
- Visual and audio analysis
- Improving application performance using parallel techniques
- Assembling the pieces
- Summary
- 2. Data Acquisition
- 3. Data Cleaning
- 4. Data Visualization
- 5. Statistical Data Analysis Techniques
- 6. Machine Learning
- 7. Neural Networks
- 8. Deep Learning
- 9. Text Analysis
- 10. Visual and Audio Analysis
- 11. Mathematical and Parallel Techniques for Data Analysis
- 12. Bringing It All Together
- 1. Getting Started with Data Science
- 2. Module 2
- 1. Applied Machine Learning Quick Start
- 2. Java Libraries and Platforms for Machine Learning
- 3. Basic Algorithms – Classification, Regression, and Clustering
- 4. Customer Relationship Prediction with Ensembles
- 5. Affinity Analysis
- 6. Recommendation Engine with Apache Mahout
- 7. Fraud and Anomaly Detection
- 8. Image Recognition with Deeplearning4j
- 9. Activity Recognition with Mobile Phone Sensors
- 10. Text Mining with Mallet – Topic Modeling and Spam Detection
- 11. What is Next?
- A. References
- 3. Module 3
- 1. Machine Learning Review
- Machine learning – history and definition
- What is not machine learning?
- Machine learning – concepts and terminology
- Machine learning – types and subtypes
- Datasets used in machine learning
- Machine learning applications
- Practical issues in machine learning
- Machine learning – roles and process
- Machine learning – tools and datasets
- Summary
- 2. Practical Approach to Real-World Supervised Learning
- Formal description and notation
- Data transformation and preprocessing
- Feature relevance analysis and dimensionality reduction
- Model building
- Model assessment, evaluation, and comparisons
- Case Study – Horse Colic Classification
- Summary
- References
- 3. Unsupervised Machine Learning Techniques
- Issues in common with supervised learning
- Issues specific to unsupervised learning
- Feature analysis and dimensionality reduction
- Clustering
- Outlier or anomaly detection
- Real-world case study
- Summary
- References
- 4. Semi-Supervised and Active Learning
- Semi-supervised learning
- Active learning
- Case study in active learning
- Summary
- References
- 5. Real-Time Stream Machine Learning
- Assumptions and mathematical notations
- Basic stream processing and computational techniques
- Concept drift and drift detection
- Incremental supervised learning
- Incremental unsupervised learning using clustering
- Modeling techniques
- Partition based
- Hierarchical based and micro clustering
- Density based
- Grid based
- Validation and evaluation techniques
- Key issues in stream cluster evaluation
- Evaluation measures
- Cluster Mapping Measures (CMM)
- V-Measure
- Other external measures
- Unsupervised learning using outlier detection
- Case study in stream learning
- Summary
- References
- 6. Probabilistic Graph Modeling
- Probability revisited
- Graph concepts
- Bayesian networks
- Representation
- Inference
- Learning
- Learning parameters
- Learning structures
- Measures to evaluate structures
- Methods for learning structures
- Constraint-based techniques
- Search and score-based techniques
- 7. Deep Learning
- Multi-layer feed-forward neural network
- Inputs, neurons, activation function, and mathematical notation
- Multi-layered neural network
- Structure and mathematical notations
- Activation functions in NN
- Training neural network
- Empirical risk minimization
- Parameter initialization
- Loss function
- Gradients
- Feed forward and backpropagation
- How does it work?
- Regularization
- L2 regularization
- L1 regularization
- Limitations of neural networks
- Deep learning
- Building blocks for deep learning
- Rectified linear activation function
- Restricted Boltzmann Machines
- Autoencoders
- Unsupervised pre-training and supervised fine-tuning
- Deep feed-forward NN
- Deep Autoencoders
- Deep Belief Networks
- Deep learning with dropouts
- Sparse coding
- Convolutional Neural Network
- CNN Layers
- Recurrent Neural Networks
- Building blocks for deep learning
- Case study
- Summary
- References
- 8. Text Mining and Natural Language Processing
- NLP, subfields, and tasks
- Text categorization
- Part-of-speech tagging (POS tagging)
- Text clustering
- Information extraction and named entity recognition
- Sentiment analysis and opinion mining
- Coreference resolution
- Word sense disambiguation
- Machine translation
- Semantic reasoning and inferencing
- Text summarization
- Automating question and answers
- Issues with mining unstructured data
- Text processing components and transformations
- Topics in text mining
- Tools and usage
- Summary
- References
- NLP, subfields, and tasks
- 9. Big Data Machine Learning – The Final Frontier
- What are the characteristics of Big Data?
- Big Data Machine Learning
- Batch Big Data Machine Learning
- Case study
- Business problem
- Machine Learning mapping
- Data collection
- Data sampling and transformation
- Spark MLlib as Big Data Machine Learning platform
- A. Linear Algebra
- B. Probability
- D. Bibliography
- Index
- Empirical risk minimization
- Multi-layer feed-forward neural network
- Modeling techniques
- 1. Machine Learning Review
Machine learning applications, especially those focused on classification, usually follow the same high-level workflow as shown in the following diagram. The workflow comprises two phases: training the classifier and classification of new instances. Both phases share common steps as you can see in the following diagram:
First, we use a set of Training data, select a representative subset as the training set, preprocess missing data, and extract features. A selected supervised learning algorithm is used to train a model, which is deployed in the second phase. The second phase puts a new data instance through the same Pre-processing and Feature extraction procedure and applies the learned model to obtain the instance label. If you are able to collect new labeled data, periodically rerun the learning phase to retrain the model, and replace the old one with the retrained one in the classification phase.
Structured data, such as transactional, customer, analytical, and market data, usually resides within a local relational database. Given a query language, such as SQL, we can query the data used for processing, as shown in the workflow in the previous diagram. Usually, all the data can be stored in the memory and further processed with a machine learning library such as Weka, Java-ML, or MALLET.
A common practice in the architecture design is to create data pipelines, where different steps in the workflow are split. For instance, in order to create a client data record, we might have to scrap the data from different data sources. The record can be then saved in an intermediate database for further processing.
To understand how the high-level aspects of big data architecture differ, let's first clarify when is the data considered big?
Big data existed long before the phrase was invented, for instance, banks and stock exchanges have been processing billions of transactions daily for years, and airline companies companies have worldwide real-time infrastructure for operational management of passenger booking, and so on. So what is big data really? Doug Laney (2001) suggested that big data is defined by three Vs: volume, velocity, and variety. Therefore, to answer the question whether your data is big, we can translate this into the following three subquestions:
If you answered all the questions with yes, then your data is probably not big, do not worry, you have just simplified your application architecture.
If your answer to all the questions was no, then your data is big! However, if you have mixed answers, then it's complicated. Some may argue that a V is important, other may say the other Vs. From the machine learning point of view, there is a fundamental difference in algorithm implementation to process the data in memory or from distributed storage. Therefore, a rule of thumb is as follows: if you cannot store your data in the memory, then you should look into a big data machine learning library.
The exact answer depends on the problem that you are trying to solve. If you're starting a new project, I'd suggest you start off with a single-machine library and prototype your algorithm, possibly with a subset of your data if the entire data does not fit into the memory. Once you've established good initial results, consider moving to something more heavy duty such as Mahout or Spark.
Big data, such as documents, weblogs, social networks, sensor data, and others, are stored in a NoSQL database, such as MongoDB, or a distributed filesystem, such as HDFS. In case we deal with structured data, we can deploy database capabilities using systems such as Cassandra or HBase built atop Hadoop. Data processing follows the MapReduce paradigm, which breaks data processing problems into smaller subproblems and distributes tasks across processing nodes. Machine learning models are finally trained with machine learning libraries such as Mahout and Spark.
Note
MongoDB is a NoSQL database, which stores documents in a JSON-like format. Read more about it at https://www.mongodb.org.
Hadoop is a framework for distributed processing of large datasets across a cluster of computers. It includes its own filesystem format HDFS, job scheduling framework YARD, and implements the MapReduce approach for parallel data processing. More about Hadoop is available at http://hadoop.apache.org/.
Cassandra is a distributed database management system build to provide fault-tolerant, scalable, and decentralized storage. More information is available at http://cassandra.apache.org/.
HBase is another database that focuses on random read/write access to distributed storage. More information is available at https://hbase.apache.org/.
-
No Comment