How to do it...

The necessary steps include the following:

  1. Insert the following imports to the project:
import java.io.StringReader;
import java.util.List;
import edu.stanford.nlp.ling.CoreLabel;
import edu.stanford.nlp.process.CoreLabelTokenFactory;
import edu.stanford.nlp.process.PTBTokenizer;
import edu.stanford.nlp.process.WordToSentenceProcessor;
  1. Add the following instance variable, which will hold the series of sentences to be processed:
private static String text = 
"We will start with a simple sentence. However, is it "
+ "possible for a sentence to end with a question "
+ "mark? Obviously that is possible! Another "
+ "complication is the use of a number such as 56.32 "
+ "or ellipses such as ... Ellipses may be found ... "
+ "with a sentence! Of course, we may also find the "
+ "use of abbreviations such as Mr. Smith or "
+ "Dr. Jones.";
  1. Add the following code to the main method:
PTBTokenizer<CoreLabel> ptbTokenizer = new PTBTokenizer<CoreLabel>(
new StringReader(text),
new CoreLabelTokenFactory(), null);
WordToSentenceProcessor<CoreLabel> wordToSentenceProcessor =
new WordToSentenceProcessor<CoreLabel>();
List<List<CoreLabel>> sentenceList =
wordToSentenceProcessor.process(ptbTokenizer.tokenize());

for (List<CoreLabel> sentence : sentenceList) {
System.out.println(sentence);
}
  1. Execute the code. You will get the following output:
[We, will, start, with, a, simple, sentence, .]
[However, ,, is, it, possible, for, a, sentence, to, end, with, a, question, mark, ?]
[Obviously, that, is, possible, !]
[Another, complication, is, the, use, of, a, number, such, as, 56.32, or, ellipses, such, as, ..., Ellipses, may, be, found, ..., with, a, sentence, !]
[Of, course, ,, we, may, also, find, the, use, of, abbreviations, such, as, Mr., Smith, or, Dr., Jones, .]
..................Content has been hidden....................

You can't read the all page of ebook, please click here login for view all page.
Reset
18.218.225.112