Natural Language Processing en Named Entity Recognition
1 like571 views
The document discusses natural language processing (NLP) and named entity recognition (NER), highlighting the steps involved in understanding human language through computers. It explains NER as the identification and classification of names within text, using historical context about Abraham Lincoln as a practical example. The document also mentions the challenges and methodologies in implementing NER systems, including rule-based and machine learning approaches.
![Wat is Named Entity Recognition?
• Named Entity Recognition is het vinden en classificeren van namen in tekst
Abraham Lincoln Listeni/ˈeɪbrəhæm ˈlɪŋkən/ (February 12, 1809 – April 15, 1865) was the 16th President of the
United States, serving from March 1861 until his assassination in April 1865. Lincoln led the United States
through its Civil War—its bloodiest war and its greatest moral, constitutional and political crisis.[1][2] In doing
so, he preserved the Union, abolished slavery, strengthened the federal government, and modernized the
economy.
Lincoln grew up on the western frontier in Kentucky and Indiana. Largely self-educated, he became a lawyer in
Illinois, a Whig Party leader, and a member of the Illinois House of Representatives, where he served from
1834 to 1846. Elected to the United States House of Representatives in 1846, Lincoln promoted rapid
modernization of the economy through banks, tariffs, and railroads. Because he had originally agreed not to
run for a second term in Congress, and his opposition to the Mexican–American War was unpopular among
Illinois voters, Lincoln returned to Springfield and resumed his successful law practice. Reentering politics in
1854, he became a leader in building the new Republican Party, which had a statewide majority in Illinois. In
1858, while taking part in a series of highly publicized debates with his opponent and rival, Democrat Stephen
A. Douglas, Lincoln spoke out against the expansion of slavery, but lost the U.S. Senate race to Douglas.
source: http://en.wikipedia.org/wiki/Abraham_Lincoln](https://image.slidesharecdn.com/dhclinics-171017141139/85/Natural-Language-Processing-en-Named-Entity-Recognition-8-320.jpg)
![Wat is Named Entity Recognition?
• Named Entity Recognition is het vinden en classificeren van namen in tekst
Abraham Lincoln Listeni/ˈeɪbrəhæm ˈlɪŋkən/ (February 12, 1809 – April 15, 1865) was the 16th President of the
United States, serving from March 1861 until his assassination in April 1865. Lincoln led the United States
through its Civil War—its bloodiest war and its greatest moral, constitutional and political crisis.[1][2] In doing
so, he preserved the Union, abolished slavery, strengthened the federal government, and modernized the
economy.
Lincoln grew up on the western frontier in Kentucky and Indiana. Largely self-educated, he became a lawyer in
Illinois, a Whig Party leader, and a member of the Illinois House of Representatives, where he served from
1834 to 1846. Elected to the United States House of Representatives in 1846, Lincoln promoted rapid
modernization of the economy through banks, tariffs, and railroads. Because he had originally agreed not to
run for a second term in Congress, and his opposition to the Mexican–American War was unpopular among
Illinois voters, Lincoln returned to Springfield and resumed his successful law practice. Reentering politics in
1854, he became a leader in building the new Republican Party, which had a statewide majority in Illinois. In
1858, while taking part in a series of highly publicized debates with his opponent and rival, Democrat Stephen
A. Douglas, Lincoln spoke out against the expansion of slavery, but lost the U.S. Senate race to Douglas.
source: http://en.wikipedia.org/wiki/Abraham_Lincoln](https://image.slidesharecdn.com/dhclinics-171017141139/85/Natural-Language-Processing-en-Named-Entity-Recognition-9-320.jpg)
![Wat is Named Entity Recognition?
• Named Entity Recognition is het vinden en classificeren van namen in tekst
Abraham Lincoln Listeni/ˈeɪbrəhæm ˈlɪŋkən/ (February 12, 1809 – April 15, 1865) was the 16th President of the
United States, serving from March 1861 until his assassination in April 1865. Lincoln led the United States
through its Civil War—its bloodiest war and its greatest moral, constitutional and political crisis.[1][2] In doing
so, he preserved the Union, abolished slavery, strengthened the federal government, and modernized the
economy.
Lincoln grew up on the western frontier in Kentucky and Indiana. Largely self-educated, he became a lawyer in
Illinois, a Whig Party leader, and a member of the Illinois House of Representatives, where he served from
1834 to 1846. Elected to the United States House of Representatives in 1846, Lincoln promoted rapid
modernization of the economy through banks, tariffs, and railroads. Because he had originally agreed not to
run for a second term in Congress, and his opposition to the Mexican–American War was unpopular among
Illinois voters, Lincoln returned to Springfield and resumed his successful law practice. Reentering politics in
1854, he became a leader in building the new Republican Party, which had a statewide majority in Illinois. In
1858, while taking part in a series of highly publicized debates with his opponent and rival, Democrat Stephen
A. Douglas, Lincoln spoke out against the expansion of slavery, but lost the U.S. Senate race to Douglas.
source: http://en.wikipedia.org/wiki/Abraham_Lincoln](https://image.slidesharecdn.com/dhclinics-171017141139/85/Natural-Language-Processing-en-Named-Entity-Recognition-10-320.jpg)
![Wat is Named Entity Recognition?
• Named Entity Recognition is het vinden en classificeren van namen in tekst
Abraham Lincoln Listeni/ˈeɪbrəhæm ˈlɪŋkən/ (February 12, 1809 – April 15, 1865) was the 16th President of the
United States, serving from March 1861 until his assassination in April 1865. Lincoln led the United States
through its Civil War—its bloodiest war and its greatest moral, constitutional and political crisis.[1][2] In doing
so, he preserved the Union, abolished slavery, strengthened the federal government, and modernized the
economy.
Lincoln grew up on the western frontier in Kentucky and Indiana. Largely self-educated, he became a lawyer in
Illinois, a Whig Party leader, and a member of the Illinois House of Representatives, where he served from
1834 to 1846. Elected to the United States House of Representatives in 1846, Lincoln promoted rapid
modernization of the economy through banks, tariffs, and railroads. Because he had originally agreed not to
run for a second term in Congress, and his opposition to the Mexican–American War was unpopular among
Illinois voters, Lincoln returned to Springfield and resumed his successful law practice. Reentering politics in
1854, he became a leader in building the new Republican Party, which had a statewide majority in Illinois. In
1858, while taking part in a series of highly publicized debates with his opponent and rival, Democrat Stephen
A. Douglas, Lincoln spoke out against the expansion of slavery, but lost the U.S. Senate race to Douglas.
source: http://en.wikipedia.org/wiki/Abraham_Lincoln](https://image.slidesharecdn.com/dhclinics-171017141139/85/Natural-Language-Processing-en-Named-Entity-Recognition-11-320.jpg)
![Wat is Named Entity Recognition?
• Named Entity Recognition is het vinden en classificeren van namen in tekst
Abraham Lincoln Listeni/ˈeɪbrəhæm ˈlɪŋkən/ (February 12, 1809 – April 15, 1865) was the 16th President of the
United States, serving from March 1861 until his assassination in April 1865. Lincoln led the United States
through its Civil War—its bloodiest war and its greatest moral, constitutional and political crisis.[1][2] In doing
so, he preserved the Union, abolished slavery, strengthened the federal government, and modernized the
economy.
Lincoln grew up on the western frontier in Kentucky and Indiana. Largely self-educated, he became a lawyer in
Illinois, a Whig Party leader, and a member of the Illinois House of Representatives, where he served from
1834 to 1846. Elected to the United States House of Representatives in 1846, Lincoln promoted rapid
modernization of the economy through banks, tariffs, and railroads. Because he had originally agreed not to
run for a second term in Congress, and his opposition to the Mexican–American War was unpopular among
Illinois voters, Lincoln returned to Springfield and resumed his successful law practice. Reentering politics in
1854, he became a leader in building the new Republican Party, which had a statewide majority in Illinois. In
1858, while taking part in a series of highly publicized debates with his opponent and rival, Democrat Stephen
A. Douglas, Lincoln spoke out against the expansion of slavery, but lost the U.S. Senate race to Douglas.
source: http://en.wikipedia.org/wiki/Abraham_Lincoln](https://image.slidesharecdn.com/dhclinics-171017141139/85/Natural-Language-Processing-en-Named-Entity-Recognition-12-320.jpg)
![Wat is Named Entity Recognition?
• Named Entity Recognition is het vinden en classificeren van namen in tekst
Abraham Lincoln Listeni/ˈeɪbrəhæm ˈlɪŋkən/ (February 12, 1809 – April 15, 1865) was the 16th President of the
United States, serving from March 1861 until his assassination in April 1865. Lincoln led the United States
through its Civil War—its bloodiest war and its greatest moral, constitutional and political crisis.[1][2] In doing
so, he preserved the Union, abolished slavery, strengthened the federal government, and modernized the
economy.
Lincoln grew up on the western frontier in Kentucky and Indiana. Largely self-educated, he became a lawyer in
Illinois, a Whig Party leader, and a member of the Illinois House of Representatives, where he served from
1834 to 1846. Elected to the United States House of Representatives in 1846, Lincoln promoted rapid
modernization of the economy through banks, tariffs, and railroads. Because he had originally agreed not to
run for a second term in Congress, and his opposition to the Mexican–American War was unpopular among
Illinois voters, Lincoln returned to Springfield and resumed his successful law practice. Reentering politics in
1854, he became a leader in building the new Republican Party, which had a statewide majority in Illinois. In
1858, while taking part in a series of highly publicized debates with his opponent and rival, Democrat Stephen
A. Douglas, Lincoln spoke out against the expansion of slavery, but lost the U.S. Senate race to Douglas.
source: http://en.wikipedia.org/wiki/Abraham_Lincoln](https://image.slidesharecdn.com/dhclinics-171017141139/85/Natural-Language-Processing-en-Named-Entity-Recognition-13-320.jpg)
![Wat is Coreference Resolution
source: http://en.wikipedia.org/wiki/Abraham_Lincoln
• Coreference resolution is het vinden van welke woorden/frasen naar hetzelfde
object verwijzen
Abraham Lincoln Listeni/ˈeɪbrəhæm ˈlɪŋkən/ (February 12, 1809 – April 15, 1865) was the 16th President of the
United States, serving from March 1861 until his assassination in April 1865. Lincoln led the United States
through its Civil War—its bloodiest war and its greatest moral, constitutional and political crisis.[1][2] In doing
so, he preserved the Union, abolished slavery, strengthened the federal government, and modernized the
economy.
Lincoln grew up on the western frontier in Kentucky and Indiana. Largely self-educated, he became a lawyer in
Illinois, a Whig Party leader, and a member of the Illinois House of Representatives, where he served from
1834 to 1846. Elected to the United States House of Representatives in 1846, Lincoln promoted rapid
modernization of the economy through banks, tariffs, and railroads. Because he had originally agreed not to
run for a second term in Congress, and his opposition to the Mexican–American War was unpopular among
Illinois voters, Lincoln returned to Springfield and resumed his successful law practice. Reentering politics in
1854, he became a leader in building the new Republican Party, which had a statewide majority in Illinois. In
1858, while taking part in a series of highly publicized debates with his opponent and rival, Democrat Stephen
A. Douglas, Lincoln spoke out against the expansion of slavery, but lost the U.S. Senate race to Douglas.](https://image.slidesharecdn.com/dhclinics-171017141139/85/Natural-Language-Processing-en-Named-Entity-Recognition-22-320.jpg)
![Maar het is eigenlijk nog ingewikkelder
source: http://en.wikipedia.org/wiki/Abraham_Lincoln
• Coreference resolution is het vinden van welke woorden/frasen naar hetzelfde
object verwijzen
Abraham Lincoln Listeni/ˈeɪbrəhæm ˈlɪŋkən/ (February 12, 1809 – April 15, 1865) was the 16th President of the
United States, serving from March 1861 until his assassination in April 1865. Lincoln led the United States
through its Civil War—its bloodiest war and its greatest moral, constitutional and political crisis.[1][2] In doing
so, he preserved the Union, abolished slavery, strengthened the federal government, and modernized the
economy.
Lincoln grew up on the western frontier in Kentucky and Indiana. Largely self-educated, he became a lawyer in
Illinois, a Whig Party leader, and a member of the Illinois House of Representatives, where he served from
1834 to 1846. Elected to the United States House of Representatives in 1846, Lincoln promoted rapid
modernization of the economy through banks, tariffs, and railroads. Because he had originally agreed not to
run for a second term in Congress, and his opposition to the Mexican–American War was unpopular among
Illinois voters, Lincoln returned to Springfield and resumed his successful law practice. Reentering politics in
1854, he became a leader in building the new Republican Party, which had a statewide majority in Illinois. In
1858, while taking part in a series of highly publicized debates with his opponent and rival, Democrat Stephen
A. Douglas, Lincoln spoke out against the expansion of slavery, but lost the U.S. Senate race to Douglas.](https://image.slidesharecdn.com/dhclinics-171017141139/85/Natural-Language-Processing-en-Named-Entity-Recognition-23-320.jpg)
![Maar het is eigenlijk nog ingewikkelder
source: http://en.wikipedia.org/wiki/Abraham_Lincoln
• Coreference resolution is het vinden van welke woorden/frasen naar hetzelfde
object verwijzen
Abraham Lincoln Listeni/ˈeɪbrəhæm ˈlɪŋkən/ (February 12, 1809 – April 15, 1865) was the 16th President of the
United States, serving from March 1861 until his assassination in April 1865. Lincoln led the United States
through its Civil War—its bloodiest war and its greatest moral, constitutional and political crisis.[1][2] In doing
so, he preserved the Union, abolished slavery, strengthened the federal government, and modernized the
economy.
Lincoln grew up on the western frontier in Kentucky and Indiana. Largely self-educated, he became a lawyer in
Illinois, a Whig Party leader, and a member of the Illinois House of Representatives, where he served from
1834 to 1846. Elected to the United States House of Representatives in 1846, Lincoln promoted rapid
modernization of the economy through banks, tariffs, and railroads. Because he had originally agreed not to
run for a second term in Congress, and his opposition to the Mexican–American War was unpopular among
Illinois voters, Lincoln returned to Springfield and resumed his successful law practice. Reentering politics in
1854, he became a leader in building the new Republican Party, which had a statewide majority in Illinois. In
1858, while taking part in a series of highly publicized debates with his opponent and rival, Democrat Stephen
A. Douglas, Lincoln spoke out against the expansion of slavery, but lost the U.S. Senate race to Douglas.](https://image.slidesharecdn.com/dhclinics-171017141139/85/Natural-Language-Processing-en-Named-Entity-Recognition-24-320.jpg)
Natural Language Processing en Named Entity Recognition
- 1. Natural Language Processing & Named Entity Recognition marieke.van.erp@huygens.knaw.nl merpeltje
- 6. Natural Language Processing • Het verwerken van natuurlijke (menselijke) taal door computers • Doel: ‘begrijpen’ van tekst • In de praktijk: verrijken van collecties, verbeteren zoekmachines, automatisch vertalen • Begrijpen van tekst wordt voor computer opgedeeld in verschillende lagen/stapjes: Tokenisation Lexical Analysis Syntactic Analysis Semantic Analysis Pragmatic Analysis Input text Speaker's intended meaning
- 7. Natural Language Processing Stappen Tokenisation Lexical Analysis Syntactic Analysis Semantic Analysis Pragmatic Analysis Input text Speaker's intended meaning • Tokenisation: detectie van zinnen en woorden • Lexicale analyse: woordsoorten (voorzetsel, adjectief) en vervoegingen • Syntactische analyse: zinsstructuur (onderwerp, persoonsvorm, lijdend voorwerp) • Semantische analyse: naar wie wordt verwezen in de tekst, waar gaat de tekst eigenlijk over? • Pragmatische analyse: wat bedoelt de spreker
- 8. Wat is Named Entity Recognition? • Named Entity Recognition is het vinden en classificeren van namen in tekst Abraham Lincoln Listeni/ˈeɪbrəhæm ˈlɪŋkən/ (February 12, 1809 – April 15, 1865) was the 16th President of the United States, serving from March 1861 until his assassination in April 1865. Lincoln led the United States through its Civil War—its bloodiest war and its greatest moral, constitutional and political crisis.[1][2] In doing so, he preserved the Union, abolished slavery, strengthened the federal government, and modernized the economy. Lincoln grew up on the western frontier in Kentucky and Indiana. Largely self-educated, he became a lawyer in Illinois, a Whig Party leader, and a member of the Illinois House of Representatives, where he served from 1834 to 1846. Elected to the United States House of Representatives in 1846, Lincoln promoted rapid modernization of the economy through banks, tariffs, and railroads. Because he had originally agreed not to run for a second term in Congress, and his opposition to the Mexican–American War was unpopular among Illinois voters, Lincoln returned to Springfield and resumed his successful law practice. Reentering politics in 1854, he became a leader in building the new Republican Party, which had a statewide majority in Illinois. In 1858, while taking part in a series of highly publicized debates with his opponent and rival, Democrat Stephen A. Douglas, Lincoln spoke out against the expansion of slavery, but lost the U.S. Senate race to Douglas. source: http://en.wikipedia.org/wiki/Abraham_Lincoln
- 9. Wat is Named Entity Recognition? • Named Entity Recognition is het vinden en classificeren van namen in tekst Abraham Lincoln Listeni/ˈeɪbrəhæm ˈlɪŋkən/ (February 12, 1809 – April 15, 1865) was the 16th President of the United States, serving from March 1861 until his assassination in April 1865. Lincoln led the United States through its Civil War—its bloodiest war and its greatest moral, constitutional and political crisis.[1][2] In doing so, he preserved the Union, abolished slavery, strengthened the federal government, and modernized the economy. Lincoln grew up on the western frontier in Kentucky and Indiana. Largely self-educated, he became a lawyer in Illinois, a Whig Party leader, and a member of the Illinois House of Representatives, where he served from 1834 to 1846. Elected to the United States House of Representatives in 1846, Lincoln promoted rapid modernization of the economy through banks, tariffs, and railroads. Because he had originally agreed not to run for a second term in Congress, and his opposition to the Mexican–American War was unpopular among Illinois voters, Lincoln returned to Springfield and resumed his successful law practice. Reentering politics in 1854, he became a leader in building the new Republican Party, which had a statewide majority in Illinois. In 1858, while taking part in a series of highly publicized debates with his opponent and rival, Democrat Stephen A. Douglas, Lincoln spoke out against the expansion of slavery, but lost the U.S. Senate race to Douglas. source: http://en.wikipedia.org/wiki/Abraham_Lincoln
- 10. Wat is Named Entity Recognition? • Named Entity Recognition is het vinden en classificeren van namen in tekst Abraham Lincoln Listeni/ˈeɪbrəhæm ˈlɪŋkən/ (February 12, 1809 – April 15, 1865) was the 16th President of the United States, serving from March 1861 until his assassination in April 1865. Lincoln led the United States through its Civil War—its bloodiest war and its greatest moral, constitutional and political crisis.[1][2] In doing so, he preserved the Union, abolished slavery, strengthened the federal government, and modernized the economy. Lincoln grew up on the western frontier in Kentucky and Indiana. Largely self-educated, he became a lawyer in Illinois, a Whig Party leader, and a member of the Illinois House of Representatives, where he served from 1834 to 1846. Elected to the United States House of Representatives in 1846, Lincoln promoted rapid modernization of the economy through banks, tariffs, and railroads. Because he had originally agreed not to run for a second term in Congress, and his opposition to the Mexican–American War was unpopular among Illinois voters, Lincoln returned to Springfield and resumed his successful law practice. Reentering politics in 1854, he became a leader in building the new Republican Party, which had a statewide majority in Illinois. In 1858, while taking part in a series of highly publicized debates with his opponent and rival, Democrat Stephen A. Douglas, Lincoln spoke out against the expansion of slavery, but lost the U.S. Senate race to Douglas. source: http://en.wikipedia.org/wiki/Abraham_Lincoln
- 11. Wat is Named Entity Recognition? • Named Entity Recognition is het vinden en classificeren van namen in tekst Abraham Lincoln Listeni/ˈeɪbrəhæm ˈlɪŋkən/ (February 12, 1809 – April 15, 1865) was the 16th President of the United States, serving from March 1861 until his assassination in April 1865. Lincoln led the United States through its Civil War—its bloodiest war and its greatest moral, constitutional and political crisis.[1][2] In doing so, he preserved the Union, abolished slavery, strengthened the federal government, and modernized the economy. Lincoln grew up on the western frontier in Kentucky and Indiana. Largely self-educated, he became a lawyer in Illinois, a Whig Party leader, and a member of the Illinois House of Representatives, where he served from 1834 to 1846. Elected to the United States House of Representatives in 1846, Lincoln promoted rapid modernization of the economy through banks, tariffs, and railroads. Because he had originally agreed not to run for a second term in Congress, and his opposition to the Mexican–American War was unpopular among Illinois voters, Lincoln returned to Springfield and resumed his successful law practice. Reentering politics in 1854, he became a leader in building the new Republican Party, which had a statewide majority in Illinois. In 1858, while taking part in a series of highly publicized debates with his opponent and rival, Democrat Stephen A. Douglas, Lincoln spoke out against the expansion of slavery, but lost the U.S. Senate race to Douglas. source: http://en.wikipedia.org/wiki/Abraham_Lincoln
- 12. Wat is Named Entity Recognition? • Named Entity Recognition is het vinden en classificeren van namen in tekst Abraham Lincoln Listeni/ˈeɪbrəhæm ˈlɪŋkən/ (February 12, 1809 – April 15, 1865) was the 16th President of the United States, serving from March 1861 until his assassination in April 1865. Lincoln led the United States through its Civil War—its bloodiest war and its greatest moral, constitutional and political crisis.[1][2] In doing so, he preserved the Union, abolished slavery, strengthened the federal government, and modernized the economy. Lincoln grew up on the western frontier in Kentucky and Indiana. Largely self-educated, he became a lawyer in Illinois, a Whig Party leader, and a member of the Illinois House of Representatives, where he served from 1834 to 1846. Elected to the United States House of Representatives in 1846, Lincoln promoted rapid modernization of the economy through banks, tariffs, and railroads. Because he had originally agreed not to run for a second term in Congress, and his opposition to the Mexican–American War was unpopular among Illinois voters, Lincoln returned to Springfield and resumed his successful law practice. Reentering politics in 1854, he became a leader in building the new Republican Party, which had a statewide majority in Illinois. In 1858, while taking part in a series of highly publicized debates with his opponent and rival, Democrat Stephen A. Douglas, Lincoln spoke out against the expansion of slavery, but lost the U.S. Senate race to Douglas. source: http://en.wikipedia.org/wiki/Abraham_Lincoln
- 13. Wat is Named Entity Recognition? • Named Entity Recognition is het vinden en classificeren van namen in tekst Abraham Lincoln Listeni/ˈeɪbrəhæm ˈlɪŋkən/ (February 12, 1809 – April 15, 1865) was the 16th President of the United States, serving from March 1861 until his assassination in April 1865. Lincoln led the United States through its Civil War—its bloodiest war and its greatest moral, constitutional and political crisis.[1][2] In doing so, he preserved the Union, abolished slavery, strengthened the federal government, and modernized the economy. Lincoln grew up on the western frontier in Kentucky and Indiana. Largely self-educated, he became a lawyer in Illinois, a Whig Party leader, and a member of the Illinois House of Representatives, where he served from 1834 to 1846. Elected to the United States House of Representatives in 1846, Lincoln promoted rapid modernization of the economy through banks, tariffs, and railroads. Because he had originally agreed not to run for a second term in Congress, and his opposition to the Mexican–American War was unpopular among Illinois voters, Lincoln returned to Springfield and resumed his successful law practice. Reentering politics in 1854, he became a leader in building the new Republican Party, which had a statewide majority in Illinois. In 1858, while taking part in a series of highly publicized debates with his opponent and rival, Democrat Stephen A. Douglas, Lincoln spoke out against the expansion of slavery, but lost the U.S. Senate race to Douglas. source: http://en.wikipedia.org/wiki/Abraham_Lincoln
- 14. Waarom zijn entiteiten zo belangrijk? • Entiteiten kunnen worden geïndexeerd • Sentiment kan aan entiteiten gekoppeld worden • Entiteiten vormen het startpunt voor veel andere informatie-extractietaken bron: https://trends.google.nl/trends/topcharts
- 15. Entity recognition voor (literatuur)collecties
- 16. Entity recognition voor (literatuur)collecties
- 17. Hoe gaat Named Entity Recognition in zijn werk? • Handgeschreven regels (bv. Persoonsnaam: “Meneer” gevolgd door woord dat begint met hoofdletter) • Hoge precisie, maar veel werk • Wat te doen met woorden die ambigu zijn? Apple, Handel, De Leeuw • Statistische (machine learning) methodes • Veel geannoteerde training data nodig • Semi-supervised machine learning wordt onderzocht om dit op te lossen
- 18. State-of-the-Art *ACL: Association for Computational Linguistics
- 19. Valkuilen • Hoge resultaten zijn uitsluitend op teksten ‘in domain’ teksten • Meeste systemen zijn geoptimaliseerd voor hedendaagse kranten • Taalvariatie (tijd, dialect, status, type publicatie) beïnvloeden resultaten • Advies: do not try this at home
- 20. Als je er toch mee aan de slag wilt • https://gate.ac.uk/ • https://nlp.stanford.edu/software/CRF-NER.shtml • https://opennlp.apache.org/ • http://ixa2.si.ehu.es/ ixa-pipes/ • https://languagemachines.github.io/frog/
- 21. Evalueer & vraag hulp • Controleer of het NER systeem de gewenste output geeft • Bijvoorbeeld door de output met de hand na te lopen • Schakel een taaltechnoloog in! • Computational Linguistics in the Netherlands • http://www.let.rug.nl/vannoord/Clin/
- 22. Wat is Coreference Resolution source: http://en.wikipedia.org/wiki/Abraham_Lincoln • Coreference resolution is het vinden van welke woorden/frasen naar hetzelfde object verwijzen Abraham Lincoln Listeni/ˈeɪbrəhæm ˈlɪŋkən/ (February 12, 1809 – April 15, 1865) was the 16th President of the United States, serving from March 1861 until his assassination in April 1865. Lincoln led the United States through its Civil War—its bloodiest war and its greatest moral, constitutional and political crisis.[1][2] In doing so, he preserved the Union, abolished slavery, strengthened the federal government, and modernized the economy. Lincoln grew up on the western frontier in Kentucky and Indiana. Largely self-educated, he became a lawyer in Illinois, a Whig Party leader, and a member of the Illinois House of Representatives, where he served from 1834 to 1846. Elected to the United States House of Representatives in 1846, Lincoln promoted rapid modernization of the economy through banks, tariffs, and railroads. Because he had originally agreed not to run for a second term in Congress, and his opposition to the Mexican–American War was unpopular among Illinois voters, Lincoln returned to Springfield and resumed his successful law practice. Reentering politics in 1854, he became a leader in building the new Republican Party, which had a statewide majority in Illinois. In 1858, while taking part in a series of highly publicized debates with his opponent and rival, Democrat Stephen A. Douglas, Lincoln spoke out against the expansion of slavery, but lost the U.S. Senate race to Douglas.
- 23. Maar het is eigenlijk nog ingewikkelder source: http://en.wikipedia.org/wiki/Abraham_Lincoln • Coreference resolution is het vinden van welke woorden/frasen naar hetzelfde object verwijzen Abraham Lincoln Listeni/ˈeɪbrəhæm ˈlɪŋkən/ (February 12, 1809 – April 15, 1865) was the 16th President of the United States, serving from March 1861 until his assassination in April 1865. Lincoln led the United States through its Civil War—its bloodiest war and its greatest moral, constitutional and political crisis.[1][2] In doing so, he preserved the Union, abolished slavery, strengthened the federal government, and modernized the economy. Lincoln grew up on the western frontier in Kentucky and Indiana. Largely self-educated, he became a lawyer in Illinois, a Whig Party leader, and a member of the Illinois House of Representatives, where he served from 1834 to 1846. Elected to the United States House of Representatives in 1846, Lincoln promoted rapid modernization of the economy through banks, tariffs, and railroads. Because he had originally agreed not to run for a second term in Congress, and his opposition to the Mexican–American War was unpopular among Illinois voters, Lincoln returned to Springfield and resumed his successful law practice. Reentering politics in 1854, he became a leader in building the new Republican Party, which had a statewide majority in Illinois. In 1858, while taking part in a series of highly publicized debates with his opponent and rival, Democrat Stephen A. Douglas, Lincoln spoke out against the expansion of slavery, but lost the U.S. Senate race to Douglas.
- 24. Maar het is eigenlijk nog ingewikkelder source: http://en.wikipedia.org/wiki/Abraham_Lincoln • Coreference resolution is het vinden van welke woorden/frasen naar hetzelfde object verwijzen Abraham Lincoln Listeni/ˈeɪbrəhæm ˈlɪŋkən/ (February 12, 1809 – April 15, 1865) was the 16th President of the United States, serving from March 1861 until his assassination in April 1865. Lincoln led the United States through its Civil War—its bloodiest war and its greatest moral, constitutional and political crisis.[1][2] In doing so, he preserved the Union, abolished slavery, strengthened the federal government, and modernized the economy. Lincoln grew up on the western frontier in Kentucky and Indiana. Largely self-educated, he became a lawyer in Illinois, a Whig Party leader, and a member of the Illinois House of Representatives, where he served from 1834 to 1846. Elected to the United States House of Representatives in 1846, Lincoln promoted rapid modernization of the economy through banks, tariffs, and railroads. Because he had originally agreed not to run for a second term in Congress, and his opposition to the Mexican–American War was unpopular among Illinois voters, Lincoln returned to Springfield and resumed his successful law practice. Reentering politics in 1854, he became a leader in building the new Republican Party, which had a statewide majority in Illinois. In 1858, while taking part in a series of highly publicized debates with his opponent and rival, Democrat Stephen A. Douglas, Lincoln spoke out against the expansion of slavery, but lost the U.S. Senate race to Douglas.
- 25. Waarom is coreference resolutie belangrijk? • Coreference is een frequent taalfenomeen • Coreference resolution is essentieel om een tekst goed te begrijpen • Coreference resolution is moeilijk omdat het over de grenzen van syntax, semantiek en discours gaat • Huidige state-of-the-art: 60~70% accuraat
- 26. Intussen in Semantic Web-land
- 27. Grounding Entities in the LOD cloud “Abraham Lincoln” http://dbpedia.org/resource/Abraham_Lincoln
- 29. Waar het mis kan gaan Paris
- 30. Waar het mis kan gaan ParisImage source: https://static.pexels.com/photos/161853/eiffel-tower-paris- france-tower-161853.jpeg
- 31. Waar het mis kan gaan ParisImage source: https://static.pexels.com/photos/161853/eiffel-tower-paris- france-tower-161853.jpeg https://upload.wikimedia.org/ wikipedia/commons/5/58/ Paris_Hilton_2%2C_2012.jpg
- 32. Waar het mis kan gaan ParisImage source: https://static.pexels.com/photos/161853/eiffel-tower-paris- france-tower-161853.jpeg https://upload.wikimedia.org/ wikipedia/commons/5/58/ Paris_Hilton_2%2C_2012.jpg https://commons.wikimedia.org/ wiki/File:Helene_Paris_David.jpg
- 33. Waar het mis kan gaan ParisImage source: https://static.pexels.com/photos/161853/eiffel-tower-paris- france-tower-161853.jpeg https://upload.wikimedia.org/ wikipedia/commons/5/58/ Paris_Hilton_2%2C_2012.jpg https://commons.wikimedia.org/ wiki/File:Helene_Paris_David.jpg https://upload.wikimedia.org/wikipedia/commons/4/49/ Eiffel_Tower_Replica_Paris_Texas_DSC_0602_ad.JPG
- 34. Waar het mis kan gaan ParisImage source: https://static.pexels.com/photos/161853/eiffel-tower-paris- france-tower-161853.jpeg https://upload.wikimedia.org/ wikipedia/commons/5/58/ Paris_Hilton_2%2C_2012.jpg https://commons.wikimedia.org/ wiki/File:Helene_Paris_David.jpg https://upload.wikimedia.org/wikipedia/commons/4/49/ Eiffel_Tower_Replica_Paris_Texas_DSC_0602_ad.JPG
- 35. Waar het mis kan gaan ParisImage source: https://static.pexels.com/photos/161853/eiffel-tower-paris- france-tower-161853.jpeg https://upload.wikimedia.org/ wikipedia/commons/5/58/ Paris_Hilton_2%2C_2012.jpg https://commons.wikimedia.org/ wiki/File:Helene_Paris_David.jpg https://upload.wikimedia.org/wikipedia/commons/4/49/ Eiffel_Tower_Replica_Paris_Texas_DSC_0602_ad.JPG
- 36. Maar je moet wel weten welke ‘Lincoln’ je moet hebben
- 38. Hoe gaat Named Entity Disambiguation in zijn werk? • Er is een database van entiteiten met context • Entiteiten worden geïdentificeerd in tekst • Het systeem zoekt per gevonden entiteit in de tekst een passende entiteit in de database • Matching op woorden • Matching op context • Ranking (vaak naar populariteit van entiteit in de database)
- 39. State-of-the-Art • Lehman et al. (2010): 80,6 • TAC KBP Entity Linking*: persoon, organisatie, locatie • Nieuwsartikelen, blogs en fora (Usenet) * entity linking en entity disambiguation worden vaak door elkaar gebruikt
- 40. Valkuilen • Vooral op ‘populaire’ entiteiten (beroemd genoeg voor Wikipedia) • Evaluatiedatasets vaak niet gebalanceerd (Van Erp et al. 2016) • Moeilijk in te richten voor eigen thesauri
- 41. Als je er zelf mee aan de slag wilt • https://github.com/dbpedia-spotlight/dbpedia-spotlight/wiki • http://nerd.eurecom.fr • FOX http://aksw.org/Projects/FOX.html • https://www.ibm.com/watson/services/natural- language-understanding/ • http://babelfy.org/
- 42. Evalueer & vraag hulp • Controleer of het NEL systeem de gewenste output geeft • Staan de entiteiten die voor mij relevant zijn wel in de knowledge base? • Schakel een Semantic Web expert in! • http://nl.dbpedia.org/web/ (Nederlandstalige DBpedia ) • https://iswc2017.semanticweb.org/ • Platform Linked Data Nederland: http://www.pilod.nl/ • https://2017.semantics.cc/
- 43. Recap • Named entity recognition en disambiguatie zijn potentiële verrijkingen van databronnen • Vaak nog onderzoeksstadium • Domeinadaptatie blijft een punt van aandacht • Maar er is wel veel interesse in taaltechnologie en semantic web gemeenschap om dit te verbeteren!
- 44. Verder lezen • Lehmann, J., Monahan, S., Nezda, L., Jung, A., & Shi, Y. (2010). LCC Approaches to Knowledge Base Population at TAC 2010. In TAC. • van Erp, M., Mendes, P. N., Paulheim, H., Ilievski, F., Plu, J., Rizzo, G., & Waitelonis, J. (2016). Evaluating Entity Linking: An Analysis of Current Benchmark Datasets and a Roadmap for Doing a Better Job. In LREC (Vol. 5, p. 2016). • Maynard, D., Bontcheva, K., & Augenstein, I. (2016) Natural Language Processing for the Semantic Web. Morgan Claypool. • Introductiefilmpje Semantic Web: https://www.youtube.com/watch? v=TJfrNo3Z-DU (Metaweb is opgekocht door Google, wat misschien iets te maken heeft met de gestructureerde zoekresultaten die je nu terugkrijgt die Seth noemde)