Valeriia Mozharova and Natalia Loukachevitch - Combining Knowledge and CRF-based Approach to Named Entity Recognition in Russian
Download as PPTX, PDF1 like479 views
1) The document describes a system for named entity recognition in Russian texts that combines a knowledge-based approach with a CRF classifier. 2) The system uses token-based, lexicon-based, and context-based features as inputs to the CRF classifier. It was tested on two open Russian text collections and achieved F-scores over 94% for person recognition. 3) The system was also evaluated using two different labeling schemes (IO and BIO) and incorporating additional entity types beyond persons, achieving higher F-scores with BIO labeling.
![Named entity recognition task
A named entity is a word or a word collocation that means a
specific object or an event and distinguishes it from other similar
objects.
1. Президент [Владимир Путин] PER 17 декабря провел
традиционную пресс-конференцию перед Новым Годом.
2. Студенты и Татьяны получат эксклюзивный пропуск на
Главный каток страны.
Methods:
1. Machine learning
2. Rule-based approach
3. Combination
2](https://image.slidesharecdn.com/mozharovacombiningknowledgeandcrf-basedapproach-160427210414/85/Valeriia-Mozharova-and-Natalia-Loukachevitch-Combining-Knowledge-and-CRF-based-Approach-to-Named-Entity-Recognition-in-Russian-2-320.jpg)
![Features based on lexicons
We used vocabularies that store lists of useful
expressions (words or phrase)
Sources:
• Phonebook
• Wikipedia
• Thesaurus (РуТез)
Single feature for each lexicon
Example:
«Набережные[geo2] Челны[geo2]»
8](https://image.slidesharecdn.com/mozharovacombiningknowledgeandcrf-basedapproach-160427210414/85/Valeriia-Mozharova-and-Natalia-Loukachevitch-Combining-Knowledge-and-CRF-based-Approach-to-Named-Entity-Recognition-in-Russian-8-320.jpg)
Valeriia Mozharova and Natalia Loukachevitch - Combining Knowledge and CRF-based Approach to Named Entity Recognition in Russian
- 1. Combining Knowledge and CRF-based Approach to Named Entity Recognition in Russian Mozharova V. A., Loukachevitch N. V. Lomonosov Moscow State University
- 2. Named entity recognition task A named entity is a word or a word collocation that means a specific object or an event and distinguishes it from other similar objects. 1. Президент [Владимир Путин] PER 17 декабря провел традиционную пресс-конференцию перед Новым Годом. 2. Студенты и Татьяны получат эксклюзивный пропуск на Главный каток страны. Methods: 1. Machine learning 2. Rule-based approach 3. Combination 2
- 3. Related works • English • A lot of works • Evaluations (MUC, CoNLL, ACE …) • Russian • Machine learning (CRF) • (Antonova, Soloviev, 2013; Podobryaev, 2013; Gareev, 2013) • Most works on own collections • Rule-based • (Trofimov, 2014) • Open collection “Person-1000” 3
- 4. Outline Our approach: CRF-based Named Entity recognition Features: • Token-based • Lexicon-based • Context-based Labeling representation • IO-scheme • BIO-scheme Experiments on open collections • Persons-1000 • Persons-1111-F (Eastern names) 4
- 5. CRF-based machine learning CRF is a tool for labeling sequential data. • CRF++ (open source implementation) Preprocessing • Morphological analyzer (POS-tagging, lemmatization, gender and grammatical case tagging) 5
- 6. Scheme of text processing 6 Text Feature Extraction: -token-based -lexicon-based -context-based Name Extraction CRF
- 7. Token features Most traditional features 1. Token initial form (lemma) 2. Number of symbols in a token 3. Letter case: BigBig, BigSmall, SmallSmall, Fence 4. Token type • part of speech • type of punctuation 5. The presence of a vowel (a binary feature) 6. If a token contains a known letter n-gram from a pre- defined set: • Кузнецов, Матвиенко, Джугашвили • Госдепартамент, Газпром 7
- 8. Features based on lexicons We used vocabularies that store lists of useful expressions (words or phrase) Sources: • Phonebook • Wikipedia • Thesaurus (РуТез) Single feature for each lexicon Example: «Набережные[geo2] Челны[geo2]» 8
- 9. lexicons . 9 Vocabulary Size, objects Clarification Examples Famous persons 31482 Famous people Владимир Путин First names 2773 First names Василий, Анна, Том Surnames 66108 Surnames Кузнецов, Грибоедов Verbs of informing 1729 Verbs that usually occur with persons высказать, признаться Companies 33380 Organization names Сбербанк Company types 6774 Organization types организация, авиафирма Geography 8969 Geographical objects Балтийское море Equipment 44094 Devices, equipment, tools устройство, телефон
- 10. Context features and example 10 Token Lemma Register Token Type Second Name Geo Label В В Small Auxiliary False False NO России РОССИЯ BigSmall Noun False Geo1 GEOPOLIT Алиев АЛИЕВ BigSmall Noun Sname1 False PER третий ТРЕТИЙ Small Numeral False False NO раз РАЗ Small Auxiliary False False NO
- 11. Expert labeling. Brat annotatiоn tool 11
- 12. Labeling representation IO-scheme (Inside-Outside) • I - belongs to named entity • O - does not belong to named entity |C| + 1 classes 12 Token IO-Labels BIO-labels Владимир I-PER B-PER Путин I-PER I-PER посетил OUTSIDE OUTSIDE Англию I-GEOPOLIT B-GEOPOLIT BIO-scheme (Begin-Inside- Outside) • B - named entity beginning • I - named entity continuation • O - not named entity 2*|C| + 1 classes
- 13. IO-labeling: aggregation of tokens into named entities 13 I-PER Person I-PER Person I-PER Person I-PER Петр Person Person I-PER Петр I-PER Person I-PER
- 14. IO-labeling: aggregation of tokens into named entities 14 I-ORG Organization I-ORG I-PER X1 … Person I-PER X1 … OUTSIDE X1 Person … Person
- 15. Target metric intersectionCount is the number of named entities labeled by both: the classier and the expert; classifierCount is the number of named entities labeled by only the classier; expertCount is the number of named entities labeled by only the expert. 15
- 16. Text collections • "Persons-1000" (1000 news documents) • Russian names: Александр Игнатенко, Алексей Волков • " Persons-1111F" (1111 news documents) • Eastern names: Абдалла Халаф, Иттё Ито We additionally labeled: • Organizations (ORG) • Media organizations having a specific function of information providing (MEDIA) • Locations (LOC) • States and capitals in the role of a state (GEOPOLIT) 16
- 17. Experiments on Collection “Persons-1000” NE Type F-score, % IO IO + rules BIO PER 94.95 95.09 96.08 ORG 80.03 80.23 83.84 LOC 92.60 92.60 94.57 Average 89.54 89.67 91.71 17 NE Type F-score, % IO IO + rules BIO PER 94.95 95.01 95.63 ORG 75.90 76.16 80.06 MEDIA 87.95 87.95 87.99 LOC 84.53 84.53 86.91 GEOPOLIT 94.65 94.65 94.50 Average 88.21 88.37 89.93 Cross-validation 3:1
- 18. Experiments on collection with Eastern names (Persons-1111F) Person name extraction “Persons-1000”: cross-validation 3:1 “Persons-1111F” : training on “Persons-1000” 18 Collection F-score, % Rule-based (Trofimov, 2014) Our system Pesons-1000 96.62 96.08 Persons-1111F 64.43 81.68
- 19. Conclusion • We presented the system for Russian Named Entity Recognition task using knowledge-based approach together with CRF classifier • We tested our system on two open text collections “Persons-1000” and “Persons-1111” and compare our results with rule-based system • We compared two labeling schemes for Russian texts: IO-scheme and BIO-scheme 19