Introduction to Machine Learning and Texture Analysis for Lesion Characterization
Download as PPTX, PDF2 likes890 views
The document provides an introduction to machine learning and texture analysis for lesion characterization, detailing the learning objectives and framework for utilizing these technologies in medical imaging, particularly for lung nodules. It emphasizes the importance of avoiding common pitfalls in model selection, data collection, and analysis while incorporating practical examples and applications for identifying malignancy. The content also addresses tools like KNIME and techniques such as deep learning and statistics essential for processing and visualizing imaging data.
![45
CT Gray Level Texture Analysis as
a Quantitative Imaging Biomarker
for Epidermal Growth Factor
Receptor Mutation Status in
Adenocarcinoma of the Lung.
[Ozkan E, et al. Am J Roentgenol
2015]
Radiomics](https://image.slidesharecdn.com/textureanalysis-171205101325/85/Introduction-to-Machine-Learning-and-Texture-Analysis-for-Lesion-Characterization-45-320.jpg)
Introduction to Machine Learning and Texture Analysis for Lesion Characterization
- 1. Introduction to Machine Learning and Texture Analysis for Lesion Characterization Barbaros Selnur Erdal, PhD Luciano M.S Prevedello, M.D. MPH Kevin Mader, PhD Joshy Cyriac Bram Stieltjes, M.D. PhD
- 2. Materials a.Slides i. http://bit.ly/2AfdoIe b.KNIME + Workflows + Data (zip file to extract for your own computer) i. Already installed on RSNA computers on Desktop ii.https://www.dropbox.com/s/3fcjvr0lfxfzmgd/knime_3.4.1.zip?dl=0 c.Extras i. Dataset on Kaggle 1.https://www.kaggle.com/kmader/lungnodemalignancy ii.A Kaggle Kernel (Jupyter Notebook) using Keras 1.https://www.kaggle.com/kmader/lung-node-cnn
- 3. Learning Objectives a.Review the basic principles of machine learning. b.Learn what texture analysis is and how to apply it to medical imaging. c.Understand how to combine texture analysis and machine learning for lesion classification tasks. d.Learn the how to visualize and analyze results. e.Understand how to avoid common mistakes like overfitting and incorrect model selection.
- 4. Objectives • Become valued discussion partner • Critical thinking • Learn how to collect domain knowledge well • Ask the right questions • Validation data • Don’t be too easily impressed • Too much artificial not enough intelligence in AI • Winning technical contests isn’t the same as solving clinical problems or understanding images • Don’t lose sight of the big picture • Not objectives • Become world-class machine learning experts
- 5. Outline • Introduction / Starting KNIME (Kevin) • Why ML and Texture Analysis are important? (Luciano) • Framework Overview (Kevin) • Value Prop, Decision, ML Task • Data Sources • Collecting Data - Preprocessing • Features - Basic Textures • Building Models • Features - Textures Deep Dive (Selnur) • From Textures to Deep Learning (Luciano) • Conclusion
- 6. Python might be good, but ... • Devices, Sessions, Graphs, Ops, Context Managers? • tf.Session(config = tf.ConfigProto(gpu_options = tf.GPUOptions(per_process_gpu_memory_fraction = 0.5))) • Code gets messy very quickly • Poor variable names • Minimal documentation • Custom functions / scattered .py • Multiple library versions
- 7. KNIME + Workflows • Medical workflows are complicated involving a large number of steps • We want transparent, reproducible pipelines for running analysis in research and production settings
- 8. Should I learn KNIME? • Supports • Matlab, R, Python scripts • Java code snippets • Writing your own plugins (Eclipse) • Natural Language Processing • Image Processing (full ImageJ / FIJI support, ImgLib2 integration) • Machine Learning Models (WEKA, scikit-learn, Decision Trees, PMML) • Deep Learning (DL4J, Keras model import, and full keras support coming) • JavaScript Visualization • Report Generation • Excel Input / Output • Database connectivity
- 9. • Why ML and Texture Analysis are important?
- 11. What is texture analysis gray-level cooccurrence matrix Zhang Y. MRI texture analysis in multiple sclerosis. Int J Biomed Imaging. 2012;2012:762804.
- 12. Clinical ApplicationsGlioblastoma MGMT methylation: Korfiatis P, Kline TL, Coufalova L, Lachance DH, Parney IF, Carter RE, Buckner JC, Erickson BJ. MRI texture features as biomarkers to predict MGMT methylation status in glioblastomas. Med Phys. 2016 Jun;43(6):2835. Methylated Unmethylated
- 13. Clinical Applications Glioblastoma MGMT methylation: Korfiatis P, Kline TL, Coufalova L, Lachance DH, Parney IF, Carter RE, Buckner JC, Erickson BJ. MRI texture features as biomarkers to predict MGMT methylation status in glioblastomas. Med Phys. 2016 Jun;43(6):2835. Sensitivity: 80.3% Specificity: 81.3%
- 14. Clinical Applications Ability to detect presence of EGFR mutation in patients w/ Adenocarcinoma of the Lung Ozkan E, West A, Dedelow JA, Chu BF, Zhao W, Yildiz VO, Otterson GA, Shilo K, Ghosh S, King M, White RD, Erdal BS. CT Gray-Level Texture Analysis as a Quantitative Imaging Biomarker of Epidermal Growth Factor Receptor Mutation Status in Adenocarcinoma of the Lung. AJR Am J Roentgenol. 2015 Nov;205(5):1016-25.
- 15. Texture Analysis Ozkan E, West A, Dedelow JA, Chu BF, Yildiz VO, Ghosh S, MD, Zhao W, MD, Shilo K, Otterson GA, White RD, Erdal BS. CT gray level texture analysis as a quantitative imaging biomarker for epidermal growth factor receptor mutation status in adenocarcinoma of the lung. AJR 2015
- 16. Clinical Applications AUC of 0.89 Can Quantitative CT Texture Analysis be Used to Differentiate Fat-poor Renal Angiomyolipoma from Renal Cell Carcinoma on Unenhanced CT Images? Hodgdon T, McInnes MD, Schieda N, Flood TA, Lamb L, Thornhill RE. Can Quantitative CT Texture Analysis be Used to Differentiate Fat-poor Renal Angiomyolipoma from Renal Cell Carcinoma on Unenhanced CT Images? Radiology. 2015 Sep;276(3):787-96.
- 17. Framework (machinelearningcanvas.com / louis@dorard.me)
- 18. Value Proposition • We want to help radiologists make decisions about lung nodules they see
- 19. Decision • Is the lung nodule malignant?
- 20. ML Task • Identify a given region of interest as malignant or benign • Input: • Image of suspect nodule • Output • Benign or malignant (Category)
- 21. Data Sources 1.This course a. Lung Nodules (64 x 64 tiles) i. Benign ii. Malignant b. Taken from LUNA16 Competition 2. Your own hospital a. PACS b. RIS Reports c. Pathology Reports
- 22. Data Sources
- 25. Collecting Data • This Course • Data is already prepared we just have to join it • At your hospital • Screen PACS for Chest CTs • NLP Processing of RIS for Lung Cancer Patients • KIS Analysis for Pathology Followups • Manually Labeling of Positive and Negative Nodes
- 26. Collecting Data • This Course • We turn the images into individual tiles • We then read in a list (CSV Reader) with a malignancy score for each tile • We combine them together
- 27. Features • Local patterns of pixels • 64 x 64 x 1 should be a good start (2D) • 64 x 64 x 5 would be better (3D) • HU values • Patient Age, Gender, Smoking History ?? • Not Referring Physician • Not Institution • Not Accession Number or Patient ID • Not Patient Name • Not Contrast • Not Pixel Spacing
- 29. Adding Features 1.Geometric Features a. Shape analysis b. Diameter measurements 2.Haralick Features 3.Moments 4.Patterns 5.Statistics Features 6.Histogram Features
- 30. Feature Exploration 1.Calculate image texture features for each image 2.Calculate statistics for features based on groups a. benign and malignant
- 31. Statistics 1.Independent groups t-test a. Groups the data by a variable (Malignancy in our case) b. Compares the values for selected independent variables and shows if they have a statistically significant difference and the associated p-value
- 33. Offline Evaluations • Predict the category as malignancy or not • Penalize based on category • (right or wrong) • confidence (better be low confidence and wrong, than high confidence and wrong)
- 34. Malignancy Results 1.Labels a. Positive - Malignant b. Negative - Benign 2. Scoring a. True Positives - Correctly Identified as Malignant b. True Negatives - Correctly Identified as Benign c. False Positives - Incorrectly identified As Malignant (overdiagnosed)
- 35. ROC Curve 1. True Positives - Correctly Identified as Malignant 2. True Negatives - Correctly Identified as Benign 3. False Positives - Incorrectly identified As Malignant (overdiagnosed) 4. False Negatives - Incorrectly Identified as Benign (missed cancer)
- 36. Making Predictions • Physician clicks on center of a nodule (x,y,z) in DICOM viewer • We grab an ROI based on that lesion and give it to the model • Model returns a category and confidence
- 37. Building a Model 1.Models a. Partitioning i. Training Data ii. Testing Data b. Model Selection i. Model Representation c. Scoring i. Confusion Matrix ii. R^2 iii. ROC Curve
- 38. Classification / Malignancy 1. Positive - Malignant 2.Negative - Without Contrast
- 39. Decision Trees
- 40. Start Texture Deep Dive
- 41. 41
- 42. 42
- 43. Calculations gray-level cooccurrence matrix Zhang Y. MRI texture analysis in multiple sclerosis. Int J Biomed Imaging. 2012;2012:762804.
- 44. 44
- 45. 45 CT Gray Level Texture Analysis as a Quantitative Imaging Biomarker for Epidermal Growth Factor Receptor Mutation Status in Adenocarcinoma of the Lung. [Ozkan E, et al. Am J Roentgenol 2015] Radiomics
- 46. 46 Normalization
- 48. 48 Pitfalls
- 49. Texture Analysis Workflow Example Filter FiltersFilter Filters
- 50. Start from Textures to Deep Learning
- 51. Image Analysis: Convolutional Neural Network • A subtype of Artificial Neural Network • Biologically inspired - organization of visual cortex 51Heinz Wässle. Parallel processing in the mammalian retina Nature Reviews Neuroscience 5, 747-757 Light Signal 0.7 to 1.5 million ganglion cells 96.6 million photoreceptors
- 52. Deep Learning for Images 28 x 28
- 53. Deep Learning for Images 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 28 x 28 original Conv 1
- 54. Deep Learning for Images Conv 1 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 Conv 2
- 55. Deep Learning for Images Conv 2 Max Pooling
- 56. Deep Learning for Images 818,743 Max Pooling Dense Weights Dense Activation
- 57. Deep Learning for Images Filter FiltersFilter Filters
- 58. CONVOLUTIONAL NETWORKS 4Quant | BIG IMAGE ANALYTICS Pixels Edges Object parts Object models → → →
- 59. Deep Learning with Convolutional Neural Networks Learn textures and patterns from the data
- 60. Building a Deep Model 1. Here we actually sketch out the model (from convolutions to pooling to fully-connected)
- 61. Entire Deep Pipeline Model Classifying And Scoring on the Validation Data Dividing into training and validation data https://www.kaggle.com/kmader/siim- medical-image-analysis- tutorial/discussion/31506
- 62. Training the Model 1. Training occurs by back- propagating the different between the prediction and reality 2.An epoch is one pass through the entire dataset 3.Many problems require multiple epochs of training in order to learn all of the complicated features
- 63. Validating the Model 1.Just like with the decision tree we can use the Predictor class to predict the class using the trained Neural Network. 2.We can then use the Scorer and the ROC curve to show how accurate the model
- 64. Visualizing the Model 1.We can use the DL4J Feedforward Predictor to show us what is happening inside of the model. 2.We combine this with a DataRow to Image to show the image as an X, Y image for each convolutional filter (T
- 65. Visualizing the Model 1.We can use the DL4J Feedforward Predictor to show us what is happening inside of the model. 2.We combine this with a DataRow to Image to show the image as an X, Y image for each convolutional filter (T dimension) 3.We see the different features the CNN learned (like smoothing and edges) Smoothing out Vertical Edges
- 66. More Complicated Models 1.LeNet - Created by Yann LeCun (http://yann.lecun.com/ exdb/lenet/) 2.We need to add an additional layer at the end to make it classify into two categories instead of 500
- 67. Improving Models 1.Tweaking hyperparameters 2.From AUC 0.71 to 0.89 (massive improvement)
- 68. Shortcomings 1. Dataset Issues ○ Are all slices really nodules? ○ 2D? ○ Resolution ○ Scanner Type 2. Model Issues ○ 2D ○ 64 x 64 input ○ Negative samples
- 69. Tips for Success 1. Interdisciplinary teams 2. Focus on your strengths 3. Ask the right questions What can go wrong https://www.kaggle.com/kmader/simple-nn-with-keras Overfitting Catastrophic Forgetting
- 70. Continuing your education 1. Kaggle ○ Open Datasets ■ Segment Lungs ■ Classify Nodules ■ Find tumors ○ Competitions 2. Luke Oakden-Rayner’s Blog (MD-PhD) https://lukeoakdenrayner.wordpress.com/
- 71. Every person looks a little bit different Every scan looks a little bit different Every scanner makes a slightly different image Every physician marks a slightly different region