In this recipe, we will create an instance of the OpenNLP SimpleTokenizer class to illustrate tokenization. We will use its tokenize method against a sample text.
- Title Page
- Copyright and Credits
- About Packt
- Contributors
- Preface
- Preparing Text for Analysis and Tokenization
- Technical requirements
- Tokenization using the Java SDK
- Tokenization using OpenNLP
- Tokenization using maximum entropy
- Training a neural network tokenizer for specialized text
- Identifying the stem of a word
- Training an OpenNLP lemmatization model
- Determining the lexical meaning of a word using OpenNLP
- Removing stop words using LingPipe
- Isolating Sentences within a Document
- Technical requirements
- Finding sentences using the Java core API
- Performing SBD using the BreakIterator class
- Using OpenNLP to perform SBD
- Using the Stanford NLP API to perform SBD
- Using the LingPipe and chunking to perform SBD
- Performing SBD on specialized text
- Training a neural network to perform SBD with specialized text
- Performing Name Entity Recognition
- Technical requirements
- Using regular expressions to find entities
- Using chunks with regular expressions to identify entities
- Using OpenNLP to find entities in text
- Isolating multiple entities types
- Using a CRF model to find entities in a document
- Using a chunker to find entities
- Training a specialized NER model
- Detecting POS Using Neural Networks
- Performing Text Classification
- Technical requirements
- Training a maximum entropy model for text classification
- Classifying documents using a maximum entropy model
- Classifying documents using the Stanford API
- Training a model to classify text using LingPipe
- Using LingPipe to classify text
- Detecting spam
- Performing sentiment analysis on reviews
- Finding Relationships within Text
- Language Identification and Translation
- Technical requirements
- Detecting the natural language in use using LingPipe
- Discovering supported languages using the Google API
- Detecting the natural language in use using the Google API
- Language translation using Google
- Language detection and translation using Amazon AWS
- Converting text to speech using the Google Cloud Text-to-Speech API
- Converting speech to text using the Google Cloud Speech-to-Text API
- Identifying Semantic Similarities within Text
- Common Text Processing and Generation Tasks
- Extracting Data for Use in NLP Analysis
- Technical requirements
- Connecting to an HTML page
- Extracting text and metadata from an HTML page
- Extracting text from a PDF document
- Extracting metadata from a PDF document
- Extracting text from a Word document
- Extracting metadata from a Word document
- Extracting text from a spreadsheet
- Extracting metadata from a spreadsheet
- Creating a Chatbot
- Installation and Configuration
- Other Books You May Enjoy
Tokenization using OpenNLP
-
No Comment
..................Content has been hidden....................
You can't read the all page of ebook, please click here login for view all page.