Challenges of NLP

The conversion of unstructured text into a machine-readable format requires careful preprocessing to preserve valuable semantic aspects of the data. How humans derive meaning from, and comprehend the content of language, is not fully understood and improving language understanding by machines remains an area of very active research.

NLP is challenging because the effective use of text data for ML requires an understanding of the inner workings of language as well as knowledge about the world to which it refers. Key challenges include the following:

  • Ambiguity due to polysemy; that is, a word or phrase can have different meanings depending on context (local high-school dropouts cut in half could be taken a couple of ways, for instance).
  • Non-standard and evolving use of language, especially in social media.
  • The use of idioms, such as throw in the towel.
  • Tricky entity names, Where is A Bug's Life playing?
  • Knowledge of the world—Mary and Sue are sisters versus Mary and Sue are mothers.
..................Content has been hidden....................

You can't read the all page of ebook, please click here login for view all page.
Reset
44.197.114.92