NLP for Biomedical Applications
Download as PPTX, PDF13 likes12,382 views
The document discusses advancements in natural language understanding (NLU) within the biomedical and clinical fields, highlighting models like BioBERT and ClinicalBERT that outperform generic models like BERT on specific tasks. It emphasizes the importance of pre-training on domain-specific data and showcases the rise of multi-modal datasets and transformer models in conversational AI. Additionally, it mentions various applications in healthcare, including text classification, sentiment analysis, and speech recognition.
NLP for Biomedical Applications
- 1. Build Cutting Edge Biomedical & Clinical NLU Models BioBERT for NLU
- 2. 2 TRENDS IN NLP & SPEECH NLP’s ImageNet Moment has Arrived You don’t need a Phd in ML to do industrial strength NLP. LOWER BARRIER TO ENTRY Textual data is still largely not utilized in healthcare, despite its value. UNSTRUCTURED & UNTAPPED Pre-train a very language model once and fine tune many times for different use cases BioBERT beats BERT on Biomedical tasks. ClinicalBERT beats BioBERT on clinical tasks. DOMAIN SPECIFIC BEATS GENERIC GROWTH OF MULTI-MODAL DATASETS Transformer & its derivatives like BERT & XLNet produce game changing performance improvements. DRAMATICALLY IMPROVING ALGORITHMS CONVERSATIONAL AI NEEDS LARGE MODELS EHR data, PubMed literature, Clinical Notes, Imaging, Devices, Patient Communications, Social Media.
- 3. 3 USE CASES IN HEALTHCARE Text Classification Sentiment Analysis Intent Classification Message Triaging Claims Processing Named Entity Recognition Information Extraction Features in ML models Knowledge Graphs Automatic Weak Labeling De-identification Question-Answer Answer questions posed in natural language Chatbots Text Summarization Summarize physician notes, radiology reports etc. Speech Recognition Call Center optimization Voice commands Machine Translation Patient Engagement Published Literature
- 4. 4 RACE TO CONVERSATIONAL AI Exceeding Human Level Performance GLUE Leaderboard Google (BERT) Facebook (RoBERTa ) Alibaba (Enriched BERT base) Uber (Plato) Microsoft (MT-DNN) Baidu (ERNIE) 2017 2018 2019 Today Google (Transformer )
- 5. 5 DOMAIN SPECIFIC BEATS GENERIC BioBERT • Pre-trained on top of BERT using PubMed data • Beats BERT on Biomedical tasks. Clinical BERT(s) • Pre-trained on top of Bio-BERT using clinical Notes • Beats BioBERT on clinical tasks.
- 7. 7
- 8. 8 https://ngc.nvidia.com/catalog/model-scripts/nvidia:biobert_for_tensorflow TRAIN USING NGC Optimized, Scalable & Easy to Use • Convenient scripts for pre-training & fine-tuning • Optimized Docker images for TensorFlow • Automatic Mixed Precision for up to 3x speedup • Scale out for pre-training & fine-tuning
- 9. 9 TRAIN USING NGC Optimized, Scalable & Easy to Use For comparison, the BioBERT paper reported 10+ days (240+ hours) to train on a 8x32 GB V100 system. https://news.developer.nvidia.com/biobert-optimized/