Challenges in Building NLP Applications in Nepali Language
6 likes1,668 views
The document discusses the challenges in building natural language processing (NLP) applications for the Nepali language, highlighting various NLP tasks and their performance levels. It emphasizes the importance of Python as a preferred programming language due to its flexibility, extensive libraries, and web framework capabilities. Key challenges include resource scarcity, unicode rendering issues, and the need for foundational research.
Challenges in Building NLP Applications in Nepali Language
- 1. CHALLENGES IN BUILDING NATURAL LANGUAGE PROCESSING APPLICATIONS FOR !पञल༠LANGUAGE - Chandan Goopta Unicode number: U+0915 HTML-code: क
- 3. NLP Task English Indic Languages Nepali Machine Translation Very Good Good Very Poor (Google/M$) Named Entity Recognition Very Good Fair None (Few Ground work) Optical Character Recognition Very Good Poor Very Poor POS Tagging Good Poor Very Poor Sentiment Analysis Very Good Fair Poor (works on-going) Speech Recognition Good Poor None (Googleâs on-work) What So Far?
- 5. SENTIMENT ANALYSIS ⢠Chunking | Sentence Chunker ⢠Tagging | POS Tagger ⢠Resources | SentiWordNet, Subjectivity WordList ⢠Machine Learning | Corpus, Tagged Samples
- 6. Build Everything from Scratch
- 7. OR I CAN USE ENGLISH LANGUAGE RESOURCES FOR NEPALI
- 8. SENTIMENT ANALYSIS ⢠Chunking | Sentence Chunker ⢠Tagging | POS Tagger ⢠Resources | SentiWordNet, Subjectivity WordList ⢠Machine Learning | Corpus, Tagged Samples
- 9. I am like Others are Like Professors are Like
- 10. BACK TO CHALLENGES ⢠Unicode Rendering in Dev-tools ⢠Lack of Resources ⢠Very Less Previous ⨠Works/Research
- 11. WHY PYTHON?
- 12. âProf. James A. Hendler⨠University of Maryland âI have the students learn Python in our undergraduate and graduate Semantic Web courses. Why? Because basically there's nothing else with the ďŹexibility and as many web librariesâ
- 13. WHY PYTHON? ⢠NLTK, although not the most efďŹcient implementation, provides a lot of awesome tools to quickly prototype a hypothesis Source: Quora
- 14. WHY PYTHON? ⢠Scipy + Numpy: Everything that isn't in NLTK is deďŹnitely in these libraries. If you want to use more advanced algorithms like Latent Semantic Indexing or Latent Dirichlet Allocation, Python has libraries to do that. Source: Quora
- 15. WHY PYTHON? ⢠Python has really great XML/HTML parsing libraries such as Beautiful Soup and Scrape.py. ⨠⨠You can use these libraries to quickly scrape the web and generate large data sets to improve the performance of your models (because lets face it, big data trumps complexity) Source: Quora
- 16. WHY PYTHON? ⢠Python has great web-frameworks like Django/ Pylons/Tornado. ⨠⨠If you invent a revolutionary sarcasm detector that can predict trends in the stock market, you can quickly integrated it into a web service, make millions, and buy a large island in a third-world country. Source: Quora
- 17. WHY PYTHON? ⢠Consider your other options: It would not make sense to use a compiled language like C++/Java for this type of work unless you needed to increase performance (computational speed, not model accuracy). ⨠⨠As far as I can tell, Ruby is completely useless for any Machine Learning, Data Mining, or Natural Language Processing task. Maybe you could use Lisp, but at this point, Python has a larger eco-system. Source: Quora
- 18. THANK YOU