Classification of Radiology Reports Using Neural Attention Models
1 like191 views
The document discusses the use of neural attention models to classify radiology reports, focusing on tasks such as assessing severity and identifying conditions like hemorrhage. It describes a dataset of annotated and unannotated radiology reports and the algorithms used, including convolutional neural networks and word embeddings. The paper emphasizes the models' performance, which aligns with human agreement scores, and explores implications for big data and model complexity in medical contexts.
Classification of Radiology Reports Using Neural Attention Models
- 1. Classification of Radiology Reports Using Neural Attention Models Bonggun Shin, Falgun H. Chokshi, Timothy Lee and Jinho D. Choi Emory University
- 2. Electronic Health Record (EHR) • EHR? • Lab values, vital sign - Structured • Clinical reports, Radiology reports - Unstructured • If all EHR records are structured • Evaluating cancer treatment outcomes • Identifying patient phenotype cohorts • Predicting clinical outcomes
- 3. Problem Definition Acute Blood Radiology Report #1 The patient has hemorrhage. Radiology Report #1 The patient has hemorrhage. Some Blood Radiology Report #3 The patient some hemorrhage. No Blood Radiology Report #7 The patient has no hemorrhage. Task 1: Severity Task 2: Blood Task 3: Mass Task 4: Stroke Task 5: Hydro
- 4. Previous Methods • Query based methods • Rules-based approach • Example • Attempt to categorize bleeding patients • The query word “hemorrhage” • Expected - The patient has hemorrhage. • False negative - “no more hemorrhage” • NLP based methods • n-grams - low performances • Neural network models • No CNN based models • Promising but black box
- 5. Neural Attention model Radiology Report Radiology Report Radiology Report Black-Box Model Normal Symptomatic Radiology Report Radiology Report Radiology Report Normal Symptomatic Explanation Other Neural Models Proposed Models
- 6. Dataset • 1400 annotated radiology head CT reports • Split into training, development, and evaluation sets (1000/200/200) • 80,000 unannotated reports for creating word embedding • Annotated according to each task
- 7. Convolutional Neural Networks Basics (1) • Tokenization: NLP4J • Train W2V: Miklov • Params • Model: CBOW, or SKIP • Dimension: 100, 200, or 400 • # of doc: 20k, 40k, 60k, or 80k (Unannotated reports) Unannotated Radiology Report #1 xxxxxxxxxxxx xxxxxxx Word2Vec
- 8. Convolutional Neural Networks Basics (2) • Each word corresponds to a respective vector • The number of tokens is set to the maximum document length The patient has hemorrhage . <pad> <pad> <pad> <pad> Radiology Report #1 The patient has hemorrhage. Word2Vec
- 9. Convolutional Neural Networks Basics (3) • Filters: 2, 3, 4, 5 • Maxpooling over each feature vector. • Dense vector is creating the softmax layer Input Matrix Feature Vectors Dense Vector Prediction
- 10. Attention vector • Filter: 1 • Maxpooling across all feature vectors. Document Matrix Attention Matrix (Filter Lenth=1) Attention Vector (MaxPool)
- 11. Embedding Attention Vector (EAV) • EAV can be seen as document vector • EAV is concatenated to the dense vector Document Matrix (Transposed) Attention Vector Embedding Attention Vector
- 13. Attention Heat Map CT HEAD W/O CONTRAST HEAD CT WITHOUT IV CONTRAST CLINICAL INDICATION : Altered mental status −1 0 1 TECHNIQUE : Axial CT images skull base vertex IV contrast . COMPARISON : Date , −1 0 1 MRI brain Date FINDINGS : Interval blooming demonstrated greater 20 foci intraparenchymal hemorrhage involving 4 −1 0 1 cerebral lobes surrounding edema . For example , intraparenchymal hemorrhage frontal lobe vertex measures 1.6 −1 0 1 1.7 cm , 1.3 1.4 cm ( series 4 image 43 ) , corpus callosum −1 0 1 hemorrhage measures measures 4.2 2.2 cm , 3.0 1.9 cm ( series 4 image 42 −1 0 1 ) , hemorrhage posterior temporal lobe measures 2.3 1.5 cm , 2.1 1.3 cm ( −1 0 1 series 4 image 34 ) . Additionally , worsening mass increasing sulcal effacement mild effacement −1 0 1 suprasellar quadrigeminal plate cisterns . No interval change low − lying tonsils . Minimal left −1 0 1 − − midline shift 2 mm. There persistent effacement lateral ventricles , unchanged size . −1 0 1 No hydrocephalus . The skull base calvarium demonstrate abnormality . Redemonstrated mucus retention cyst maxillary −1 0 1 sinus . The remaining included paranasal sinuses mastoid air cells clear . IMPRESSION : 1. −1 0 1 Interval increase size / blooming greater 20 intraparenchymal hemorrhages surrounding edema involving 4 cerebral lobes −1 0 1 . Please report details . 2. Interval worsening diffuse cerebral edema sulcal effacement , mild −1 0 1 effacement cisterns , midline shift stable low lying tonsils . 3. No acute large territory −1 0 1 infarction definite foci hemorrhage . Important findings communicated name name page info Date name name name −1 0 1 This final report , dictated radiology name name name name , agrees preliminary report dictated −1 0 1 overnight name . These images reviewed interpreted name name name , name −1 0 1 CT HEAD W/O CONTRAST HEAD CT WITHOUT IV CONTRAST CLINICAL INDICATION : Altered mental status −1 0 1 TECHNIQUE : Axial CT images skull base vertex IV contrast . COMPARISON : Date , −1 0 1 MRI brain Date FINDINGS : Interval blooming demonstrated greater 20 foci intraparenchymal hemorrhage involving 4 −1 0 1 cerebral lobes surrounding edema . For example , intraparenchymal hemorrhage frontal lobe vertex measures 1.6 −1 0 1 1.7 cm , 1.3 1.4 cm ( series 4 image 43 ) , corpus callosum −1 0 1 hemorrhage measures measures 4.2 2.2 cm , 3.0 1.9 cm ( series 4 image 42 −1 0 1 ) , hemorrhage posterior temporal lobe measures 2.3 1.5 cm , 2.1 1.3 cm ( −1 0 1 series 4 image 34 ) . Additionally , worsening mass increasing sulcal effacement mild effacement −1 0 1 suprasellar quadrigeminal plate cisterns . No interval change low − lying tonsils . Minimal left −1 0 1 − − midline shift 2 mm. There persistent effacement lateral ventricles , unchanged size . −1 0 1 No hydrocephalus . The skull base calvarium demonstrate abnormality . Redemonstrated mucus retention cyst maxillary −1 0 1 sinus . The remaining included paranasal sinuses mastoid air cells clear . IMPRESSION : 1. −1 0 1 Interval increase size / blooming greater 20 intraparenchymal hemorrhages surrounding edema involving 4 cerebral lobes −1 0 1 . Please report details . 2. Interval worsening diffuse cerebral edema sulcal effacement , mild −1 0 1 effacement cisterns , midline shift stable low lying tonsils . 3. No acute large territory −1 0 1 infarction definite foci hemorrhage . Important findings communicated name name page info Date name name name −1 0 1 This final report , dictated radiology name name name name , agrees preliminary report dictated −1 0 1 overnight name . These images reviewed interpreted name name name , name −1 0 1 Acute Blood Mass Effect
- 14. Performance Comparison on the Test data • Annotation: Two trained medical doctors • Accuracy is comparable to the human agreement scores • Both CNN and NAM outperform the baseline
- 15. Performance Trends Number of Documents for training w2v Number of Documents to train the models
- 16. Discussion • Comparable to human agreement scores • Explanatory features • Big data is good for a complex model • Annotation is expensive • data synthesis? • Transfer learning • n-gram attention?