Taweh Beysolow II
Applied Natural Language Processing with PythonImplementing Machine Learning and Deep Learning Algorithms for Natural Language Processing
Taweh Beysolow II
San Francisco, California, USA
ISBN 978-1-4842-3732-8e-ISBN 978-1-4842-3733-5
https://doi.org/10.1007/978-1-4842-3733-5
Library of Congress Control Number: 2018956300
© Taweh Beysolow II 2018
This work is subject to copyright. All rights are reserved by the Publisher, whether the whole or part of the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations, recitation, broadcasting, reproduction on microfilms or in any other physical way, and transmission or information storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology now known or hereafter developed.
Trademarked names, logos, and images may appear in this book. Rather than use a trademark symbol with every occurrence of a trademarked name, logo, or image we use the names, logos, and images only in an editorial fashion and to the benefit of the trademark owner, with no intention of infringement of the trademark. The use in this publication of trade names, trademarks, service marks, and similar terms, even if they are not identified as such, is not to be taken as an expression of opinion as to whether or not they are subject to proprietary rights.
While the advice and information in this book are believed to be true and accurate at the date of publication, neither the authors nor the editors nor the publisher can accept any legal responsibility for any errors or omissions that may be made. The publisher makes no warranty, express or implied, with respect to the material contained herein.
Distributed to the book trade worldwide by Springer Science+Business Media New York, 233 Spring Street, 6th Floor, New York, NY 10013. Phone 1-800-SPRINGER, fax (201) 348-4505, e-mail [email protected], or visit www.springeronline.com. Apress Media, LLC is a California LLC and the sole member (owner) is Springer Science + Business Media Finance Inc (SSBM Finance Inc). SSBM Finance Inc is a Delaware corporation.

To my family, friends, and colleagues for their continued support and encouragement to do more with myself than I often can conceive of doing

Introduction

Thank you for choosing Applied Natural Language Processing with Python for your journey into natural language processing (NLP). Readers should be aware that this text should not be considered a comprehensive study of machine learning, deep learning, or computer programming. As such, it is assumed that you are familiar with these techniques to some degree. Regardless, a brief review of the concepts necessary to understand the tasks that you will perform in the book is provided.

After the brief review, we begin by examining how to work with raw text data, slowly working our way through how to present data to machine learning and deep learning algorithms. After you are familiar with some basic preprocessing algorithms, we will make our way into some of the more advanced NLP tasks, such as training and working with trained word embeddings, spell-check, text generation, and question-and-answer generation.

All of the examples utilize the Python programming language and popular deep learning and machine learning frameworks, such as scikit-learn, Keras, and TensorFlow. Readers can feel free to access the source code utilized in this book on the corresponding GitHub page and/or try their own methods for solving the various problems tackled in this book with the datasets provided.

Acknowledgments

A special thanks to Santanu Pattanayak, Divya Modi, Celestin Suresh John, and everyone at Apress for the wonderful experience. It has been a pleasure to work with you all on this text. I couldn’t have asked for a better team.

Table of Contents

Chapter 1:​ What Is Natural Language Processing?​ 1
The History of Natural Language Processing 2
A Review of Machine Learning and Deep Learning 4
NLP, Machine Learning, and Deep Learning Packages with Python 4
Applications of Deep Learning to NLP 10
Summary 12
Chapter 2:​ Review of Deep Learning 13
Multilayer Perceptrons and Recurrent Neural Networks 13
Toy Example 1:​ Modeling Stock Returns with the MLP Model 15
Vanishing Gradients and Why ReLU Helps to Prevent Them 27
Loss Functions and Backpropagation 29
Recurrent Neural Networks and Long Short-Term Memory 30
Toy Example 2:​ Modeling Stock Returns with the RNN Model 32
Toy Example 3:​ Modeling Stock Returns with the LSTM Model 40
Summary 41
Chapter 3:​ Working with Raw Text 43
Tokenization and Stop Words 44
The Bag-of-Words Model (BoW) 50
CountVectorizer 51
Example Problem 1:​ Spam Detection 53
Term Frequency Inverse Document Frequency 57
Example Problem 2:​ Classifying Movie Reviews 62
Summary 74
Chapter 4:​ Topic Modeling and Word Embeddings 77
Topic Model and Latent Dirichlet Allocation (LDA) 77
Topic Modeling with LDA on Movie Review Data 81
Non-Negative Matrix Factorization (NMF) 86
Word2Vec 90
Example Problem 4.​2:​ Training a Word Embedding (Skip-Gram) 94
Continuous Bag-of-Words (CBoW) 103
Example Problem 4.​2:​ Training a Word Embedding (CBoW) 105
Global Vectors for Word Representation (GloVe) 106
Example Problem 4.​4:​ Using Trained Word Embeddings with LSTMs 111
Paragraph2Vec:​ Distributed Memory of Paragraph Vectors (PV-DM) 115
Example Problem 4.​5:​ Paragraph2Vec Example with Movie Review Data 116
Summary 118
Chapter 5:​ Text Generation, Machine Translation, and Other Recurrent Language Modeling Tasks 121
Text Generation with LSTMs 122
Bidirectional RNNs (BRNN) 126
Creating a Name Entity Recognition Tagger 128
Sequence-to-Sequence Models (Seq2Seq) 133
Question and Answer with Neural Network Models 134
Summary 141
Conclusion and Final Statements 142
Index145

About the Author and About the Technical Reviewer

About the Author

Taweh Beysolow II
../images/463133_1_En_BookFrontmatter_Figb_HTML.jpg

is a data scientist and author currently based in San Francisco, California. He has a bachelor’s degree in economics from St. Johns University and a master’s degree in applied statistics from Fordham University. His professional experience has included working at Booz Allen Hamilton, as a consultant and in various startups as a data scientist, specifically focusing on machine learning. He has applied machine learning to federal consulting, financial services, and agricultural sectors.

 

About the Technical Reviewer

Santanu Pattanayak
../images/463133_1_En_BookFrontmatter_Figc_HTML.jpg

currently works at GE Digital as a staff data scientist and is the author of the deep learning book Pro Deep Learning with TensorFlow: A Mathematical Approach to Advanced Artificial Intelligence in Python (Apress, 2017). He has more than eight years of experience in the data analytics/data science field and a background in development and database technologies. Prior to joining GE, Santanu worked at companies such as RBS, Capgemini, and IBM. He graduated with a degree in electrical engineering from Jadavpur University, Kolkata, and is an avid math enthusiast. Santanu is currently pursuing a master’s degree in data science from the Indian Institute of Technology (IIT), Hyderabad. He also devotes his time to data science hackathons and Kaggle competitions, where he ranks within the top 500 across the globe. Santanu was born and brought up in West Bengal, India, and currently resides in Bangalore, India, with his wife.

 
..................Content has been hidden....................

You can't read the all page of ebook, please click here login for view all page.
Reset
18.222.137.240