SlideShare a Scribd company logo

.NET Fest 2019. Сергей Корж. Natural Language Processing in .NET

N
NETFest

This document contains the slides for a presentation on natural language processing (NLP) with .NET. The presentation introduces common NLP tasks like analysis, transformation, and generation. It discusses NLP concepts like bag-of-words, TF-IDF, n-grams, and word embeddings. Tools for NLP with .NET are presented, including ML.NET, Catalyst, and Microsoft Recognizers libraries. Demonstrations of text summarization and document tagging using these tools are described. The presentation concludes that NLP for basic tasks is possible with .NET libraries, though features are still limited compared to other languages.

Education
1 of 37
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
Тема доклада Тема доклада Тема доклада KYIV 2019 Natural Language Processing with .NET .NET CONFERENCE #1 IN UKRAINE
Тема доклада Тема доклада Тема доклада .NET LEVEL UP About me .NET CONFERENCE #1 IN UKRAINE KYIV 2019 Sergiy Korzh 25+ years in software development 20 year running own business .NET developer since 2004 iForum.ua (technology section) Projects: EasyQuery (https://korzh.com/easyquery) Easy.Report (http://easy.report) Aistant (https://aistant.com/) Twitter: @korzhs LinkedIn: https://www.linkedin.com/in/korzh/
Тема доклада Тема доклада Тема доклада .NET LEVEL UP Agenda .NET CONFERENCE #1 IN UKRAINE KYIV 2019 1 Introduction to NLP (main tasks and basic concepts) NLP Tools for .NET (and not only)2 3 Demos 4 Useful materials and conclusions
Тема доклада Тема доклада Тема доклада .NET LEVEL UP Why NLP on .NET? .NET CONFERENCE #1 IN UKRAINE KYIV 2019
Тема доклада Тема доклада Тема доклада .NET LEVEL UP Why NLP on .NET? .NET CONFERENCE #1 IN UKRAINE KYIV 2019 Because we love .NET, right? Quick and easy (for simple NLP tasks) No “glue” code
Тема доклада Тема доклада Тема доклада .NET LEVEL UP Remarks .NET CONFERENCE #1 IN UKRAINE KYIV 2019 “Light” NLP tasks only! No Deep Learning Beginner level topics
.NET LEVEL UP NLP Tasks .NET CONFERENCE #1 IN UKRAINE KYIV 2019 1 Linguistic Analysis Transformation 2 3 Generation4
.NET LEVEL UP NLP Tasks .NET CONFERENCE #1 IN UKRAINE KYIV 2019 1 Linguistic • Segmentation • Part of speech tagging • Named-entity recognition • Relation extraction • Syntactic parsing • Coreference resolution • Semantic parsing
.NET LEVEL UP NLP Tasks’ Examples .NET CONFERENCE #1 IN UKRAINE KYIV 2019 2 Analysis • Spam-filter • Sentiment analysis • Text similarity • Information extraction
.NET LEVEL UP NLP Tasks’ Examples .NET CONFERENCE #1 IN UKRAINE KYIV 2019 3 Transformation • Machine translation • Speech to Text / Text to speech • Grammar correction • Text summarization
.NET LEVEL UP NLP Tasks’ Examples .NET CONFERENCE #1 IN UKRAINE KYIV 2019 4 Generation • Question Answering • Chat bots • Story generation
.NET LEVEL UP NLP Pipeline .NET CONFERENCE #1 IN UKRAINE KYIV 2019 TEXT Text Featurizing (Numeric representation) ML Algorithm RESULT
.NET LEVEL UP NLP Pipeline: Classic .NET CONFERENCE #1 IN UKRAINE KYIV 2019 from AYLIEN blog
.NET LEVEL UP NLP Pipeline: Deep Learning .NET CONFERENCE #1 IN UKRAINE KYIV 2019 from AYLIEN blog
.NET LEVEL UP NLP concepts: Bag of words .NET CONFERENCE #1 IN UKRAINE KYIV 2019 The way to represent your text for ML algorithms • Word frequency • One-hot encoding • TF-IDF • Other metrics Encoding approaches:
.NET LEVEL UP NLP concepts: TF-IDF .NET CONFERENCE #1 IN UKRAINE KYIV 2019 For a word-document pair, TF-IDF shows the importance of the word in the document. Used in all kinds of information retrieval tasks: • Search • Text mining • Stop-words filtering
.NET LEVEL UP NLP concepts: N-grams .NET CONFERENCE #1 IN UKRAINE KYIV 2019 Word N-grams n-gram is a contiguous sequence of n items from a given sample of text. “I live in Kyiv” word bi-grams 1. # I 2. I live 3. live in 4. in Kyiv 5. Kyiv # Character N-grams “I live in Kyiv” character bi-grams 1. #_ 2. _I 3. I_ 4. _l 5. li 6. Iv 7. ve 8. . . .
.NET LEVEL UP NLP concepts: Word Embeddings .NET CONFERENCE #1 IN UKRAINE KYIV 2019 A set of techniques which allow to map words (or phrases) to numeric vectors. The words with similar meanings have “close” vectors. word Vector man [0.23, 0.56, …] king [0.34, 0.16, …] woman [0.41, 0.73, …] queen [0.09, 0.62, …] [king] – [man] + [woman] ≈ [queen] Popular embeddings algorithms:  Word2Vec  fastText  Glove  . . .
.NET LEVEL UP NLP concepts: Language Model .NET CONFERENCE #1 IN UKRAINE KYIV 2019 allows to compute a probability of a word in a sequence. Where used? (spoiler: almost everywhere!) Please, give me a … [ pen: 0.002, example: 0.0001, hand:0.08, … ] • Machine translation • Error correction • Speech recognition • Text generation
.NET LEVEL UP NLP Tools .NET CONFERENCE #1 IN UKRAINE KYIV 2019 1 Online services Python libraries .NET Libraries 2 3 Azure Cognitive Services, IBM Watson, Amazon AI Services NLTK, spaCy, skikit-learn, gensim, Pattern ML.NET, Microsoft.Speech, Microsoft.Recognizers, Catalyst
.NET LEVEL UP .NET libs: ML.NET .NET CONFERENCE #1 IN UKRAINE KYIV 2019 https://dotnet.microsoft.com/apps/machinelearning-ai/ml-dotnet Pros: • Native for .NET (Core) • Backed my Microsoft • Super performant (at least MS says that ) • Extended with TensorFlow & more NLP features: • Text normalization • Tokenizing • N-gram • Word embeddings • Stop words removal Cons: • Poor NLP features • English-only (mostly) • Not convenient for using separately from ML pipeline
.NET LEVEL UP .NET libs: Catalyst .NET CONFERENCE #1 IN UKRAINE KYIV 2019 NLP features: • Text normalization • Tokenizing • POS-tagging • Word embeddings • Stop words removal https://github.com/curiosity-ai/catalyst Pros: • Native for .NET (Core) • Inspired by spaCy library • Fast tokenizer • Has pretrained models • Allows to train your own models (based on Universal Dependencies project) Cons: • Early beta (or even alpha). Version 0.0.2795 • English-only (mostly)
.NET LEVEL UP .NET libs: Microsoft.Recognizers .NET CONFERENCE #1 IN UKRAINE KYIV 2019 • Rule-based • Recognizes numbers, units, date/time, etc • Supports about 10 different languages • Not only .NET (JavaScript, Python, Java) • No support for Russian or Ukrainian  https://github.com/Microsoft/Recognizers-Text/
.NET LEVEL UP Other useful libraries .NET CONFERENCE #1 IN UKRAINE KYIV 2019 DEMO 1 Text summarization (extraction based) using home-brewed NLP TEXT Detect language Break into sentences Tokenize and get stems sentence1 sentence2 sentence3 stem1 1 3 5 stem2 0 2 4 stem3 3 4 0 stem4 2 0 2 Bag of words S1 S2 S3 S1 0 1.21 0.2 S2 1.21 0 3.56 S3 0.2 3.56 0 Similarity matrix Page rank algorithm Summary (top-rated sentences)
Other useful libraries
Other useful libraries
Other useful libraries
.NET LEVEL UP Other useful libraries .NET CONFERENCE #1 IN UKRAINE KYIV 2019 DEMO 2 Text summarization using ML.NET
Other useful libraries
Other useful libraries
.NET LEVEL UP Other useful libraries .NET CONFERENCE #1 IN UKRAINE KYIV 2019 DEMO 3 Document tagging (with TF-IDF and Catalyst POS tagging)
Other useful libraries
Other useful libraries
Other useful libraries
.NET LEVEL UP Useful resources .NET CONFERENCE #1 IN UKRAINE KYIV 2019 Universal Dependencies https://universaldependencies.org/ Lang-uk http://lang.org.ua/uk/ https://github.com/korzh/Korzh.NLP All source code of this talk Math.net – numerical computation algorithms for .NET https://www.mathdotnet.com/ http://tiny.cc/dotnet-nlp-libs List of .NET libraries with some NLP features
.NET LEVEL UP Conclusions .NET CONFERENCE #1 IN UKRAINE KYIV 2019 Catalyst library looks promising but still a way to go Contribute! We can do NLP on .NET (for the basic tasks at least) ML.NET library good and reliable but limited NLP features
.NET LEVEL UP Other useful libraries .NET CONFERENCE #1 IN UKRAINE KYIV 2019 Thank you! Sergiy Korzh Twitter: @korzhs LinkedIn: https://www.linkedin.com/in/korzh/ Facebook: https://www.facebook.com/sergiy.korzh Email: sergiy@korzh.com

More Related Content

PPTX
Scala vs. Python: Which Language Should be learned in 2020
NexSoftsys
 
ODP
PDQ Programming Languages plus an overview of Alice - Frank Ducrest
Matthew Turland
 
PDF
[INNOVATUBE] Tech Talk #3: Golang - Takaaki Mizuno
Nexus FrontierTech
 
PDF
About programming languages
Ganesh Samarthyam
 
PDF
MER: a Minimal Named-Entity Recognition Tagger and Annotation Server
Francisco Couto
 
PPTX
Translating software with SDL Passolo?
Loctimize GmbH
 
ODP
ANTLR4 in depth
Владимир Кожаев
 
PPTX
Android maven Road to flutter| Mavenizing Flutter for web
OluwatobiAkinpelu
 
Scala vs. Python: Which Language Should be learned in 2020
NexSoftsys
 
PDQ Programming Languages plus an overview of Alice - Frank Ducrest
Matthew Turland
 
[INNOVATUBE] Tech Talk #3: Golang - Takaaki Mizuno
Nexus FrontierTech
 
About programming languages
Ganesh Samarthyam
 
MER: a Minimal Named-Entity Recognition Tagger and Annotation Server
Francisco Couto
 
Translating software with SDL Passolo?
Loctimize GmbH
 
ANTLR4 in depth
Владимир Кожаев
 
Android maven Road to flutter| Mavenizing Flutter for web
OluwatobiAkinpelu
 

What's hot (20)

PDF
Engaging new l10n contributors through Open Source Contributhon
Youngbin Han
 
PPTX
Agile Tools for PHP
philipjting
 
PDF
Agile Localization: Oxymoron or Heroic Achievement?
Laura Dent
 
PDF
[INNOVATUBE] Tech Talk #3: Golang - Vũ Nguyễn
Nexus FrontierTech
 
PPTX
.NET Fest 2019. Алексей Голуб. Монадные парсер-комбинаторы в C# (простой спос...
NETFest
 
PDF
Kotlin & Arrow the functional way
Thoughtworks
 
PPTX
Creating multillingual apps for android
Sergi Martínez
 
PPTX
APIdays Paris 2014 - The State of Web API Languages
Restlet
 
PDF
Introduction to Algorithms and Data Structures in Swift 4: Get ready for prog...
vefusisowu
 
PDF
Building LibreOffice Korean Community and CJK common & different issues
DaeHyun Sung
 
PDF
Towards a Commons RDF Java library
Sergio Fernández
 
PPT
A First Look at Google's Go Programming Language
Ganesh Samarthyam
 
PDF
Death to project documentation with eXtreme Programming
Alex Fernandez
 
PDF
Ryan Curtin, Principal Research Scientist, Symantec at MLconf ATL 2016
MLconf
 
PDF
[LibreOffice conference 2021] The challenge of Using LibreOffice & Building L...
DaeHyun Sung
 
PDF
Automating boring and repetitive UbuCon Asia video and subtitle stuffs
Youngbin Han
 
PDF
Introduction to OmegaT
Qabiria
 
PDF
Kotlin strives for Deep Learning
Izzet Mustafaiev
 
PDF
The Go programming language - Intro by MyLittleAdventure
mylittleadventure
 
PPTX
How to Review your Translation with 2 Free and Open Source QA Tools
Qabiria
 
Engaging new l10n contributors through Open Source Contributhon
Youngbin Han
 
Agile Tools for PHP
philipjting
 
Agile Localization: Oxymoron or Heroic Achievement?
Laura Dent
 
[INNOVATUBE] Tech Talk #3: Golang - Vũ Nguyễn
Nexus FrontierTech
 
.NET Fest 2019. Алексей Голуб. Монадные парсер-комбинаторы в C# (простой спос...
NETFest
 
Kotlin & Arrow the functional way
Thoughtworks
 
Creating multillingual apps for android
Sergi Martínez
 
APIdays Paris 2014 - The State of Web API Languages
Restlet
 
Introduction to Algorithms and Data Structures in Swift 4: Get ready for prog...
vefusisowu
 
Building LibreOffice Korean Community and CJK common & different issues
DaeHyun Sung
 
Towards a Commons RDF Java library
Sergio Fernández
 
A First Look at Google's Go Programming Language
Ganesh Samarthyam
 
Death to project documentation with eXtreme Programming
Alex Fernandez
 
Ryan Curtin, Principal Research Scientist, Symantec at MLconf ATL 2016
MLconf
 
[LibreOffice conference 2021] The challenge of Using LibreOffice & Building L...
DaeHyun Sung
 
Automating boring and repetitive UbuCon Asia video and subtitle stuffs
Youngbin Han
 
Introduction to OmegaT
Qabiria
 
Kotlin strives for Deep Learning
Izzet Mustafaiev
 
The Go programming language - Intro by MyLittleAdventure
mylittleadventure
 
How to Review your Translation with 2 Free and Open Source QA Tools
Qabiria
 
Ad

Similar to .NET Fest 2019. Сергей Корж. Natural Language Processing in .NET (20)

PDF
Natural Language Processing (NLP)
Yuriy Guts
 
PDF
Training at AI Frontiers 2018 - Ni Lao: Weakly Supervised Natural Language Un...
AI Frontiers
 
PPTX
Introduction to Natural Language Processing - Stages in NLP Pipeline, Challen...
resming1
 
PPTX
Natural Language Processing (NLP).pptx
HelmandAtssar
 
PPTX
Intro to nlp
ankit_ppt
 
PDF
introtonlp-190218095523 (1).pdf
AdityaMishra178868
 
PDF
Natural Language Processing with Python
Benjamin Bengfort
 
PDF
Visual-Semantic Embeddings: some thoughts on Language
Roelof Pieters
 
PPTX
Machine learning (ML) and natural language processing (NLP)
Nikola Milosevic
 
PPTX
NLP Introduction and basics of natural language processing
mailtoahmedhassan
 
PDF
Deep Learning for Natural Language Processing: Word Embeddings
Roelof Pieters
 
PDF
Deep learning for natural language embeddings
Roelof Pieters
 
PDF
Natural language processing (NLP) introduction
Robert Lujo
 
PDF
A Gentle Introduction to Text Analysis :)
UNCResearchHub
 
PPTX
Natural language processing state of the art, current trends and challenges.pptx
wasiqbarat1
 
PPTX
Building NLP solutions for Davidson ML Group
botsplash.com
 
PDF
NLP and Knowledge Graphs
Carlos Badenes-Olmedo
 
PPTX
NLP.pptx
Rahul Borate
 
PDF
Representation Learning of Text for NLP
Anuj Gupta
 
PDF
Anthiil Inside workshop on NLP
Satyam Saxena
 
Natural Language Processing (NLP)
Yuriy Guts
 
Training at AI Frontiers 2018 - Ni Lao: Weakly Supervised Natural Language Un...
AI Frontiers
 
Introduction to Natural Language Processing - Stages in NLP Pipeline, Challen...
resming1
 
Natural Language Processing (NLP).pptx
HelmandAtssar
 
Intro to nlp
ankit_ppt
 
introtonlp-190218095523 (1).pdf
AdityaMishra178868
 
Natural Language Processing with Python
Benjamin Bengfort
 
Visual-Semantic Embeddings: some thoughts on Language
Roelof Pieters
 
Machine learning (ML) and natural language processing (NLP)
Nikola Milosevic
 
NLP Introduction and basics of natural language processing
mailtoahmedhassan
 
Deep Learning for Natural Language Processing: Word Embeddings
Roelof Pieters
 
Deep learning for natural language embeddings
Roelof Pieters
 
Natural language processing (NLP) introduction
Robert Lujo
 
A Gentle Introduction to Text Analysis :)
UNCResearchHub
 
Natural language processing state of the art, current trends and challenges.pptx
wasiqbarat1
 
Building NLP solutions for Davidson ML Group
botsplash.com
 
NLP and Knowledge Graphs
Carlos Badenes-Olmedo
 
NLP.pptx
Rahul Borate
 
Representation Learning of Text for NLP
Anuj Gupta
 
Anthiil Inside workshop on NLP
Satyam Saxena
 
Ad

More from NETFest (20)

PDF
.NET Fest 2019. Николай Балакин. Микрооптимизации в мире .NET
NETFest
 
PPTX
.NET Fest 2019. Сергей Калинец. Efficient Microservice Communication with .NE...
NETFest
 
PPTX
.NET Fest 2019. Оля Гавриш. .NET Core 3.0 и будущее .NET
NETFest
 
PPTX
.NET Fest 2019. Оля Гавриш. Машинное обучение для .NET программистов
NETFest
 
PPTX
.NET Fest 2019. Roberto Freato. Provisioning Azure PaaS fluently with Managem...
NETFest
 
PPTX
.NET Fest 2019. Halil Ibrahim Kalkan. Implementing Domain Driven Design
NETFest
 
PPTX
.NET Fest 2019. Сергій Бута. Feature Toggles: Dynamic Configuration at Wirex
NETFest
 
PPTX
.NET Fest 2019. Michael Staib. Hot Chocolate: GraphQL Schema Stitching with A...
NETFest
 
PPTX
.NET Fest 2019. Андрей Литвинов. Async lifetime tests with xUnit and AutoFixture
NETFest
 
PPTX
.NET Fest 2019. Анатолий Колесник. Love, Death & F# Tests
NETFest
 
PPTX
.NET Fest 2019. Roberto Freato. Azure App Service deep dive
NETFest
 
PPTX
.NET Fest 2019. Леонид Молотиевский. DotNet Core in production
NETFest
 
PPTX
.NET Fest 2019. Александр Демчук. How to measure relationships within the Com...
NETFest
 
PDF
.NET Fest 2019. Anna Melashkina та Philipp Bauknecht. Dragons in a Mixed Real...
NETFest
 
PDF
.NET Fest 2019. Alex Thissen. Architecting .NET solutions in a Docker ecosystem
NETFest
 
PPTX
.NET Fest 2019. Stas Lebedenko. Practical serverless use cases in Azure with ...
NETFest
 
PPTX
.NET Fest 2019. Сергей Медведев. How serverless makes Integration TDD a reali...
NETFest
 
PDF
.NET Fest 2019. Eran Stiller. Create Your Own Serverless PKI with .NET & Azur...
NETFest
 
PDF
.NET Fest 2019. Eran Stiller. 6 Lessons I Learned on My Journey from Monolith...
NETFest
 
PPTX
.NET Fest 2019. Kevin Dockx. Uncovering Swagger/OpenAPI
NETFest
 
.NET Fest 2019. Николай Балакин. Микрооптимизации в мире .NET
NETFest
 
.NET Fest 2019. Сергей Калинец. Efficient Microservice Communication with .NE...
NETFest
 
.NET Fest 2019. Оля Гавриш. .NET Core 3.0 и будущее .NET
NETFest
 
.NET Fest 2019. Оля Гавриш. Машинное обучение для .NET программистов
NETFest
 
.NET Fest 2019. Roberto Freato. Provisioning Azure PaaS fluently with Managem...
NETFest
 
.NET Fest 2019. Halil Ibrahim Kalkan. Implementing Domain Driven Design
NETFest
 
.NET Fest 2019. Сергій Бута. Feature Toggles: Dynamic Configuration at Wirex
NETFest
 
.NET Fest 2019. Michael Staib. Hot Chocolate: GraphQL Schema Stitching with A...
NETFest
 
.NET Fest 2019. Андрей Литвинов. Async lifetime tests with xUnit and AutoFixture
NETFest
 
.NET Fest 2019. Анатолий Колесник. Love, Death & F# Tests
NETFest
 
.NET Fest 2019. Roberto Freato. Azure App Service deep dive
NETFest
 
.NET Fest 2019. Леонид Молотиевский. DotNet Core in production
NETFest
 
.NET Fest 2019. Александр Демчук. How to measure relationships within the Com...
NETFest
 
.NET Fest 2019. Anna Melashkina та Philipp Bauknecht. Dragons in a Mixed Real...
NETFest
 
.NET Fest 2019. Alex Thissen. Architecting .NET solutions in a Docker ecosystem
NETFest
 
.NET Fest 2019. Stas Lebedenko. Practical serverless use cases in Azure with ...
NETFest
 
.NET Fest 2019. Сергей Медведев. How serverless makes Integration TDD a reali...
NETFest
 
.NET Fest 2019. Eran Stiller. Create Your Own Serverless PKI with .NET & Azur...
NETFest
 
.NET Fest 2019. Eran Stiller. 6 Lessons I Learned on My Journey from Monolith...
NETFest
 
.NET Fest 2019. Kevin Dockx. Uncovering Swagger/OpenAPI
NETFest
 

Recently uploaded (20)

PPTX
Renaissance Architecture: A Journey from Faith to Humanism
DrMonicaSharma
 
PPTX
Institutional Correction lecture only . . .
limvtheghins
 
PDF
FourierSeries-QuestionsWithAnswers(Part-A).pdf
Raji Murugan
 
PDF
BÀI TẬP BỔ TRỢ 4 KỸ NĂNG TIẾNG ANH 9 GLOBAL SUCCESS - CẢ NĂM - BÁM SÁT FORM Đ...
Nguyen Thanh Tu Collection
 
PDF
Abdominal Access Techniques with Prof. Dr. R K Mishra
mohitsuren827
 
PDF
Sports Quiz easy sports quiz sports quiz
nikunj2757
 
PDF
Module 4: Burden of Disease Tutorial Slides S2 2025
Jonathan Hallett
 
PDF
Classroom Observation Tools for Teachers
Nicaramirez
 
PDF
grade 11-chemistry_fetena_net_5883.pdf teacher guide for all student
banteamlakmebratu738
 
PDF
O7-L3 Supply Chain Operations - ICLT Program
labyankof
 
PDF
Insiders guide to clinical Medicine.pdf
AbhishekPatil315128
 
PDF
Pre independence Education in Inndia.pdf
goofy5603
 
PDF
Microbial disease of the cardiovascular and lymphatic systems
KeyaSharma
 
PDF
Basic Mud Logging Guide for educational purpose
wdandworks
 
PPTX
school management -TNTEU- B.Ed., Semester II Unit 1.pptx
NihumathunnisaH
 
PPTX
human mycosis Human fungal infections are called human mycosis..pptx
shilpa939953
 
PDF
The Lost Whites of Pakistan by Jahanzaib Mughal.pdf
jahanzaibmughal6
 
PDF
Anesthesia in Laparoscopic Surgery in India
mohitsuren827
 
PDF
Complications of Minimal Access Surgery at WLH
mohitsuren827
 
PDF
Saundersa Comprehensive Review for the NCLEX-RN Examination.pdf
sreejithcareers
 
Renaissance Architecture: A Journey from Faith to Humanism
DrMonicaSharma
 
Institutional Correction lecture only . . .
limvtheghins
 
FourierSeries-QuestionsWithAnswers(Part-A).pdf
Raji Murugan
 
BÀI TẬP BỔ TRỢ 4 KỸ NĂNG TIẾNG ANH 9 GLOBAL SUCCESS - CẢ NĂM - BÁM SÁT FORM Đ...
Nguyen Thanh Tu Collection
 
Abdominal Access Techniques with Prof. Dr. R K Mishra
mohitsuren827
 
Sports Quiz easy sports quiz sports quiz
nikunj2757
 
Module 4: Burden of Disease Tutorial Slides S2 2025
Jonathan Hallett
 
Classroom Observation Tools for Teachers
Nicaramirez
 
grade 11-chemistry_fetena_net_5883.pdf teacher guide for all student
banteamlakmebratu738
 
O7-L3 Supply Chain Operations - ICLT Program
labyankof
 
Insiders guide to clinical Medicine.pdf
AbhishekPatil315128
 
Pre independence Education in Inndia.pdf
goofy5603
 
Microbial disease of the cardiovascular and lymphatic systems
KeyaSharma
 
Basic Mud Logging Guide for educational purpose
wdandworks
 
school management -TNTEU- B.Ed., Semester II Unit 1.pptx
NihumathunnisaH
 
human mycosis Human fungal infections are called human mycosis..pptx
shilpa939953
 
The Lost Whites of Pakistan by Jahanzaib Mughal.pdf
jahanzaibmughal6
 
Anesthesia in Laparoscopic Surgery in India
mohitsuren827
 
Complications of Minimal Access Surgery at WLH
mohitsuren827
 
Saundersa Comprehensive Review for the NCLEX-RN Examination.pdf
sreejithcareers
 

.NET Fest 2019. Сергей Корж. Natural Language Processing in .NET

  • 1. Тема доклада Тема доклада Тема доклада KYIV 2019 Natural Language Processing with .NET .NET CONFERENCE #1 IN UKRAINE
  • 2. Тема доклада Тема доклада Тема доклада .NET LEVEL UP About me .NET CONFERENCE #1 IN UKRAINE KYIV 2019 Sergiy Korzh 25+ years in software development 20 year running own business .NET developer since 2004 iForum.ua (technology section) Projects: EasyQuery (https://korzh.com/easyquery) Easy.Report (http://easy.report) Aistant (https://aistant.com/) Twitter: @korzhs LinkedIn: https://www.linkedin.com/in/korzh/
  • 3. Тема доклада Тема доклада Тема доклада .NET LEVEL UP Agenda .NET CONFERENCE #1 IN UKRAINE KYIV 2019 1 Introduction to NLP (main tasks and basic concepts) NLP Tools for .NET (and not only)2 3 Demos 4 Useful materials and conclusions
  • 4. Тема доклада Тема доклада Тема доклада .NET LEVEL UP Why NLP on .NET? .NET CONFERENCE #1 IN UKRAINE KYIV 2019
  • 5. Тема доклада Тема доклада Тема доклада .NET LEVEL UP Why NLP on .NET? .NET CONFERENCE #1 IN UKRAINE KYIV 2019 Because we love .NET, right? Quick and easy (for simple NLP tasks) No “glue” code
  • 6. Тема доклада Тема доклада Тема доклада .NET LEVEL UP Remarks .NET CONFERENCE #1 IN UKRAINE KYIV 2019 “Light” NLP tasks only! No Deep Learning Beginner level topics
  • 7. .NET LEVEL UP NLP Tasks .NET CONFERENCE #1 IN UKRAINE KYIV 2019 1 Linguistic Analysis Transformation 2 3 Generation4
  • 8. .NET LEVEL UP NLP Tasks .NET CONFERENCE #1 IN UKRAINE KYIV 2019 1 Linguistic • Segmentation • Part of speech tagging • Named-entity recognition • Relation extraction • Syntactic parsing • Coreference resolution • Semantic parsing
  • 9. .NET LEVEL UP NLP Tasks’ Examples .NET CONFERENCE #1 IN UKRAINE KYIV 2019 2 Analysis • Spam-filter • Sentiment analysis • Text similarity • Information extraction
  • 10. .NET LEVEL UP NLP Tasks’ Examples .NET CONFERENCE #1 IN UKRAINE KYIV 2019 3 Transformation • Machine translation • Speech to Text / Text to speech • Grammar correction • Text summarization
  • 11. .NET LEVEL UP NLP Tasks’ Examples .NET CONFERENCE #1 IN UKRAINE KYIV 2019 4 Generation • Question Answering • Chat bots • Story generation
  • 12. .NET LEVEL UP NLP Pipeline .NET CONFERENCE #1 IN UKRAINE KYIV 2019 TEXT Text Featurizing (Numeric representation) ML Algorithm RESULT
  • 13. .NET LEVEL UP NLP Pipeline: Classic .NET CONFERENCE #1 IN UKRAINE KYIV 2019 from AYLIEN blog
  • 14. .NET LEVEL UP NLP Pipeline: Deep Learning .NET CONFERENCE #1 IN UKRAINE KYIV 2019 from AYLIEN blog
  • 15. .NET LEVEL UP NLP concepts: Bag of words .NET CONFERENCE #1 IN UKRAINE KYIV 2019 The way to represent your text for ML algorithms • Word frequency • One-hot encoding • TF-IDF • Other metrics Encoding approaches:
  • 16. .NET LEVEL UP NLP concepts: TF-IDF .NET CONFERENCE #1 IN UKRAINE KYIV 2019 For a word-document pair, TF-IDF shows the importance of the word in the document. Used in all kinds of information retrieval tasks: • Search • Text mining • Stop-words filtering
  • 17. .NET LEVEL UP NLP concepts: N-grams .NET CONFERENCE #1 IN UKRAINE KYIV 2019 Word N-grams n-gram is a contiguous sequence of n items from a given sample of text. “I live in Kyiv” word bi-grams 1. # I 2. I live 3. live in 4. in Kyiv 5. Kyiv # Character N-grams “I live in Kyiv” character bi-grams 1. #_ 2. _I 3. I_ 4. _l 5. li 6. Iv 7. ve 8. . . .
  • 18. .NET LEVEL UP NLP concepts: Word Embeddings .NET CONFERENCE #1 IN UKRAINE KYIV 2019 A set of techniques which allow to map words (or phrases) to numeric vectors. The words with similar meanings have “close” vectors. word Vector man [0.23, 0.56, …] king [0.34, 0.16, …] woman [0.41, 0.73, …] queen [0.09, 0.62, …] [king] – [man] + [woman] ≈ [queen] Popular embeddings algorithms:  Word2Vec  fastText  Glove  . . .
  • 19. .NET LEVEL UP NLP concepts: Language Model .NET CONFERENCE #1 IN UKRAINE KYIV 2019 allows to compute a probability of a word in a sequence. Where used? (spoiler: almost everywhere!) Please, give me a … [ pen: 0.002, example: 0.0001, hand:0.08, … ] • Machine translation • Error correction • Speech recognition • Text generation
  • 20. .NET LEVEL UP NLP Tools .NET CONFERENCE #1 IN UKRAINE KYIV 2019 1 Online services Python libraries .NET Libraries 2 3 Azure Cognitive Services, IBM Watson, Amazon AI Services NLTK, spaCy, skikit-learn, gensim, Pattern ML.NET, Microsoft.Speech, Microsoft.Recognizers, Catalyst
  • 21. .NET LEVEL UP .NET libs: ML.NET .NET CONFERENCE #1 IN UKRAINE KYIV 2019 https://dotnet.microsoft.com/apps/machinelearning-ai/ml-dotnet Pros: • Native for .NET (Core) • Backed my Microsoft • Super performant (at least MS says that ) • Extended with TensorFlow & more NLP features: • Text normalization • Tokenizing • N-gram • Word embeddings • Stop words removal Cons: • Poor NLP features • English-only (mostly) • Not convenient for using separately from ML pipeline
  • 22. .NET LEVEL UP .NET libs: Catalyst .NET CONFERENCE #1 IN UKRAINE KYIV 2019 NLP features: • Text normalization • Tokenizing • POS-tagging • Word embeddings • Stop words removal https://github.com/curiosity-ai/catalyst Pros: • Native for .NET (Core) • Inspired by spaCy library • Fast tokenizer • Has pretrained models • Allows to train your own models (based on Universal Dependencies project) Cons: • Early beta (or even alpha). Version 0.0.2795 • English-only (mostly)
  • 23. .NET LEVEL UP .NET libs: Microsoft.Recognizers .NET CONFERENCE #1 IN UKRAINE KYIV 2019 • Rule-based • Recognizes numbers, units, date/time, etc • Supports about 10 different languages • Not only .NET (JavaScript, Python, Java) • No support for Russian or Ukrainian  https://github.com/Microsoft/Recognizers-Text/
  • 24. .NET LEVEL UP Other useful libraries .NET CONFERENCE #1 IN UKRAINE KYIV 2019 DEMO 1 Text summarization (extraction based) using home-brewed NLP TEXT Detect language Break into sentences Tokenize and get stems sentence1 sentence2 sentence3 stem1 1 3 5 stem2 0 2 4 stem3 3 4 0 stem4 2 0 2 Bag of words S1 S2 S3 S1 0 1.21 0.2 S2 1.21 0 3.56 S3 0.2 3.56 0 Similarity matrix Page rank algorithm Summary (top-rated sentences)
  • 28. .NET LEVEL UP Other useful libraries .NET CONFERENCE #1 IN UKRAINE KYIV 2019 DEMO 2 Text summarization using ML.NET
  • 31. .NET LEVEL UP Other useful libraries .NET CONFERENCE #1 IN UKRAINE KYIV 2019 DEMO 3 Document tagging (with TF-IDF and Catalyst POS tagging)
  • 35. .NET LEVEL UP Useful resources .NET CONFERENCE #1 IN UKRAINE KYIV 2019 Universal Dependencies https://universaldependencies.org/ Lang-uk http://lang.org.ua/uk/ https://github.com/korzh/Korzh.NLP All source code of this talk Math.net – numerical computation algorithms for .NET https://www.mathdotnet.com/ http://tiny.cc/dotnet-nlp-libs List of .NET libraries with some NLP features
  • 36. .NET LEVEL UP Conclusions .NET CONFERENCE #1 IN UKRAINE KYIV 2019 Catalyst library looks promising but still a way to go Contribute! We can do NLP on .NET (for the basic tasks at least) ML.NET library good and reliable but limited NLP features
  • 37. .NET LEVEL UP Other useful libraries .NET CONFERENCE #1 IN UKRAINE KYIV 2019 Thank you! Sergiy Korzh Twitter: @korzhs LinkedIn: https://www.linkedin.com/in/korzh/ Facebook: https://www.facebook.com/sergiy.korzh Email: sergiy@korzh.com

Editor's Notes

  • #22: What kind of normalization? How to get tokens? What n-gramming is supported (word, character?) What kind of word embeddings? Only English? How to add my own stop-word removal?