- Python Text Processing with NLTK 2.0 Cookbook
- Table of Contents
- Python Text Processing with NLTK 2.0 Cookbook
- Credits
- About the Author
- About the Reviewers
- Preface
- 1. Tokenizing Text and WordNet Basics
- Introduction
- Tokenizing text into sentences
- Tokenizing sentences into words
- Tokenizing sentences using regular expressions
- Filtering stopwords in a tokenized sentence
- Looking up synsets for a word in WordNet
- Looking up lemmas and synonyms in WordNet
- Calculating WordNet synset similarity
- Discovering word collocations
- 2. Replacing and Correcting Words
- 3. Creating Custom Corpora
- Introduction
- Setting up a custom corpus
- Creating a word list corpus
- Creating a part-of-speech tagged word corpus
- Creating a chunked phrase corpus
- Creating a categorized text corpus
- Creating a categorized chunk corpus reader
- Lazy corpus loading
- Creating a custom corpus view
- Creating a MongoDB backed corpus reader
- Corpus editing with file locking
- 4. Part-of-Speech Tagging
- Introduction
- Default tagging
- Training a unigram part-of-speech tagger
- Combining taggers with backoff tagging
- Training and combining Ngram taggers
- Creating a model of likely word tags
- Tagging with regular expressions
- Affix tagging
- Training a Brill tagger
- Training the TnT tagger
- Using WordNet for tagging
- Tagging proper names
- Classifier based tagging
- 5. Extracting Chunks
- Introduction
- Chunking and chinking with regular expressions
- Merging and splitting chunks with regular expressions
- Expanding and removing chunks with regular expressions
- Partial parsing with regular expressions
- Training a tagger-based chunker
- Classification-based chunking
- Extracting named entities
- Extracting proper noun chunks
- Extracting location chunks
- Training a named entity chunker
- 6. Transforming Chunks and Trees
- 7. Text Classification
- Introduction
- Bag of Words feature extraction
- Training a naive Bayes classifier
- Training a decision tree classifier
- Training a maximum entropy classifier
- Measuring precision and recall of a classifier
- Calculating high information words
- Combining classifiers with voting
- Classifying with multiple binary classifiers
- 8. Distributed Processing and Handling Large Datasets
- Introduction
- Distributed tagging with execnet
- Distributed chunking with execnet
- Parallel list processing with execnet
- Storing a frequency distribution in Redis
- Storing a conditional frequency distribution in Redis
- Storing an ordered dictionary in Redis
- Distributed word scoring with Redis and execnet
- 9. Parsing Specific Data
- A. Penn Treebank Part-of-Speech Tags
- Index
In this book, you will find a number of styles of text that distinguish between different kinds of information. Here are some examples of these styles, and an explanation of their meaning.
Code words in text are shown as follows: "Now we want to split para into sentences. First we need to import the sentence tokenization function, and then we can call it with the paragraph as an argument."
A block of code is set as follows:
>>> para = "Hello World. It's good to see you. Thanks for buying this book." >>> from nltk.tokenize import sent_tokenize >>> sent_tokenize(para)
New terms and important words are shown in bold.
-
No Comment
..................Content has been hidden....................
You can't read the all page of ebook, please click here login for view all page.