Transfer Learning for Natural Language Processing
by
Transfer Learning for Natural Language Processing
- Transfer Learning for Natural Language Processing
- Copyright
- dedication
- contents
- front matter
- Part 1 Introduction and overview
- 1 What is transfer learning?
- 1.1 Overview of representative NLP tasks
- 1.2 Understanding NLP in the context of AI
- 1.2.1 Artificial intelligence (AI)
- 1.2.2 Machine learning
- 1.2.3 Natural language processing (NLP)
- 1.3 A brief history of NLP advances
- 1.3.1 General overview
- 1.3.2 Recent transfer learning advances
- 1.4 Transfer learning in computer vision
- 1.4.1 General overview
- 1.4.2 Pretrained ImageNet models
- 1.4.3 Fine-tuning pretrained ImageNet models
- 1.5 Why is NLP transfer learning an exciting topic to study now?
- Summary
- 2 Getting started with baselines: Data preprocessing
- 2.1 Preprocessing email spam classification example data
- 2.1.1 Loading and visualizing the Enron corpus
- 2.1.2 Loading and visualizing the fraudulent email corpus
- 2.1.3 Converting the email text into numbers
- 2.2 Preprocessing movie sentiment classification example data
- 2.3 Generalized linear models
- 2.3.1 Logistic regression
- 2.3.2 Support vector machines (SVMs)
- Summary
- 3 Getting started with baselines: Benchmarking and optimization
- 3.1 Decision-tree-based models
- 3.1.1 Random forests (RFs)
- 3.1.2 Gradient-boosting machines (GBMs)
- 3.2 Neural network models
- 3.2.1 Embeddings from Language Models (ELMo)
- 3.2.2 Bidirectional Encoder Representations from Transformers (BERT)
- 3.3 Optimizing performance
- 3.3.1 Manual hyperparameter tuning
- 3.3.2 Systematic hyperparameter tuning
- Summary
- Part 2 Shallow transfer learning and deep transfer learning with recurrent neural networks (RNNs)
- 4 Shallow transfer learning for NLP
- 5 Preprocessing data for recurrent neural network deep transfer learning experiments
- 5.1 Preprocessing tabular column-type classification data
- 5.1.1 Obtaining and visualizing tabular data
- 5.1.2 Preprocessing tabular data
- 5.1.3 Encoding preprocessed data as numbers
- 5.2 Preprocessing fact-checking example data
- 5.2.1 Special problem considerations
- 5.2.2 Loading and visualizing fact-checking data
- Summary
- 6 Deep transfer learning for NLP with recurrent neural networks
- 6.1 Semantic Inference for the Modeling of Ontologies (SIMOn)
- 6.1.1 General neural architecture overview
- 6.1.2 Modeling tabular data
- 6.1.3 Application of SIMOn to tabular column-type classification data
- 6.2 Embeddings from Language Models (ELMo)
- 6.2.1 ELMo bidirectional language modeling
- 6.2.2 Application to fake news detection
- 6.3 Universal Language Model Fine-Tuning (ULMFiT)
- 6.3.1 Target task language model fine-tuning
- 6.3.2 Target task classifier fine-tuning
- Summary
- Part 3 Deep transfer learning with transformers and adaptation strategies
- 7 Deep transfer learning for NLP with the transformer and GPT
- 7.1 The transformer
- 7.1.1 An introduction to the transformers library and attention visualization
- 7.1.2 Self-attention
- 7.1.3 Residual connections, encoder-decoder attention, and positional encoding
- 7.1.4 Application of pretrained encoder-decoder to translation
- 7.2 The Generative Pretrained Transformer
- 7.2.1 Architecture overview
- 7.2.2 Transformers pipelines introduction and application to text generation
- 7.2.3 Application to chatbots
- Summary
- 8 Deep transfer learning for NLP with BERT and multilingual BERT
- 8.1 Bidirectional Encoder Representations from Transformers (BERT)
- 8.1.1 Model architecture
- 8.1.2 Application to question answering
- 8.1.3 Application to fill in the blanks and next-sentence prediction tasks
- 8.2 Cross-lingual learning with multilingual BERT (mBERT)
- 8.2.1 Brief JW300 dataset overview
- 8.2.2 Transfer mBERT to monolingual Twi data with the pretrained tokenizer
- 8.2.3 mBERT and tokenizer trained from scratch on monolingual Twi data
- Summary
- 9 ULMFiT and knowledge distillation adaptation strategies
- 10 ALBERT, adapters, and multitask adaptation strategies
- 11 Conclusions
- 11.1 Overview of key concepts
- 11.2 Other emerging research trends
- 11.2.1 RoBERTa
- 11.2.2 GPT-3
- 11.2.3 XLNet
- 11.2.4 BigBird
- 11.2.5 Longformer
- 11.2.6 Reformer
- 11.2.7 T5
- 11.2.8 BART
- 11.2.9 XLM
- 11.2.10 TAPAS
- 11.3 Future of transfer learning in NLP
- 11.4 Ethical and environmental considerations
- 11.5 Staying up-to-date
- 11.5.1 Kaggle and Zindi competitions
- 11.5.2 arXiv
- 11.5.3 News and social media (Twitter)
- 11.6 Final words
- Summary
- appendix A Kaggle primer
- appendix B Introduction to fundamental deep learning tools
- index
Transfer Learning for Natural Language Processing
To comment go to liveBook
Manning
Shelter Island
For more information on this and other Manning titles go to
-
No Comment
..................Content has been hidden....................
You can't read the all page of ebook, please click here login for view all page.