Named Entity recognition in Sanskrit
1 like1,818 views
The document discusses named entity recognition in Sanskrit. It notes that conventional methods used for English do not work well for Sanskrit due to a lack of annotated data and linguistic style differences. The document outlines approaches used for named entity recognition in Sanskrit, including supervised approaches applied to linguistic corpora with annotated ground truths, feature selection based on natural signals in the language like capitalization in English, and using techniques like hidden Markov models and conditional random fields with a dataset from the Bagavatam and generated ground truths accounting for features like parts of speech, position, lemmas, suffixes, and context.
Named Entity recognition in Sanskrit
- 1. N a m e d E n t i t y R e c o g n i t i o n i n S a n s k r i t
- 2. I R M E Named Entities Names of places, persons, organizations etc. NER Identify Named entites in a given corpora Supervised Approaches Applied to various linguistic corpora Need annotated ground truth Feature Selection Based on natural signals used for the language Capitals in English
- 3. I R M E Ape the west Conventional methods applied for english does not work Lack of annotated data Linguistic style differences NLP Techniques Rise & Fall of HMM Success in Hindi with CRF MALLET CRF Tool Sequence tagging Dataset Bagavatam from Vedabase Ground truth generation
- 4. I R M E Ground Truth Intuition : Named entities don’t change form in any language Features POS – noun cases, verb classes Pronoun or not Features Position Lemma - enrich corpus, otherwise lost due to internal sandhi Features Suffix Word – surface form Context taken care of
- 5. I R M E