Artificial Intelligence in pathology
- 1. Dr.Neha Singh Dr.Amit K.Yadav Dr.Ravi Pratap Singh VMMC and Safdarjung Hospital,New Delhi
- 2. CONTENTS Introduction - what is AI? Machine Learning and its types Deep learning History of AI Why AI in pathology? Applications in Pathology: Advantages Challenges Take home message • Breast cancer • Lung cancer • Prostate cancer • Brain cancer • Ovarian cancer • Cervical cancer • IHC • Gene Mutation Prediction • Tumor detection for Molecular Analysis
- 3. Artificial Intelligence Artificial Intelligence Human-made • Wisdom • Capacity to learn and solve problem +
- 4. Science/Technique Of making machines do things that would require intelligence if done by human
- 6. MACHINE LEARNING • A Technique for teaching machines to learn • Focuses on the development of computer programs that can access data and use it learn for themselves. Why Machine Learning? • We cannot program everything. • Some task are difficult to define algorithmically.
- 8. Types Of Machine Learning
- 9. In machine learning, • Situations are called input. • Solutions are called target or output. Supervised • made algorithms. • For every input,there is corresponding output • Can’t think and take decisions. Unsupervised • Training machine learning task only with a set of inputs. • If any situation arises, able to find the structures, patterns and relationships between different inputs. • Ultimately make an algorithm for the situation to reach the target with maximum accuracy. • Can think and take decisions.
- 10. • Deep learning is a subset of machine learning in artificial intelligence (AI) that has networks capable of learning unsupervised from data that is unstructured or unlabeled. • Also known as deep neural learning or deep neural network Deep Learning
- 11. Learning Hierarchical Representations • Text – Character → word → word group → clause → sentence → story Low- level features output Mid- level features High-level features Trainable classifier Increasing level of abstraction
- 12. A neural network is a series of algorithms that endeavors to recognize underlying relationships in a set of data through a process that mimics the way the human brain operates. In this sense, neural networks refer to systems of neurons, either organic or artificial in nature Deep learning-mind of scientist
- 14. • American computer scientist and cognitive scientist • Developed a high level programming language - LISP • LISP was very influential in the early development of AI. • British mathematician and computer scientist • In second world war,he deviced no.of techniques to break german codes History of AI
- 15. •1965- Joseph Weizenbaum at MIT built ELIZA-A Computer Program for the Study of Natural Language Communication Between Man and Machine •1977 Edward-developed a clinical consultation system, MYCIN -could analyze infectious symptoms to derive causal bacteria and drug treatment recommendations •1992- David-Developed Pathfinder- a probabilistic reasoning expert system in the field of hematopathology– to assist pathologist to deal with uncertain knowledge.
- 16. •2003- Baker et al. commercial CAD systems(Computer-aided detection) in detecting architectural distortion of breast mammography (sensitivity 49%). • 2016- Angermueller et al. - used DL(Deep Learning) methods in genomic and biological problems such as molecular trait prediction, mutation effect prediction, and cellular image analysis.
- 17. Why AI in Pathology? Data is exactly why pathology is so well positioned to use various forms of AI.A study conducted in UK claims that in the time period of 5 years in 5 hospitals and two academic medical standards generated 500 million data. An acute shortage of pathologist in many countries which eventually increases the workload. Increased cancer screening programs resulting in increased workloads Increasing complexity of pathology tests driving up the time taken per case.
- 18. Applications of AI in Pathology
- 19. H & E slides / IHC slides Slide Scanner Whole Slide Imaging(WSI) Multiple images in form of patches (labelled/unlabelled) Data stored in database Digital Pathology
- 21. FNAC - Breast • A study by Hrushikesh et al. in 2017 at IIT Kharagpur • Data – 37 breast sample 175 images of breast FNAC • Task – Benign / Malignant • Model – NVIDIA Deep Learning GPU Training System (DIGITS) • Performance – Image based decision with accuracy of 89.7% Breast Cancer
- 23. • Li et al. 2018 in California • Data - 160 normal and 110 mets LN WSIs for training 81 normal and 49 mets LN WSIs for testing • Model - Neural conditional random field (NCRF). Two major components: CNN and CRF. CNN - acts as a feature extractor, that takes a grid of patches as input encodes each patch as a fixed-length vector representation (i.e. embedding). CRF - takes the grid of embeddings as input models their spatial correlations. Decision – Yes / No Performance – Patch level decision with acc. 93.8% Lymph Node Metastasis in case of breast cancer
- 26. • Rannen Triki et al. (2018) • Data-4,921 frozen section frames • Task – Benign / Cancer • Model – CNN • Performance - Patch level decision acc. 94.96% Fig:Optical Coherence Tomography (OCT) with DNN Intraoperative margin detection of breast lump
- 28. Mitotic detection in Breast cancer • Cires¸ an et al. (2013) • Data - MITOS/300 mitosis in 50 images • Task – Mitosis detection • Model – CNN • Performance - Detection F1- score 0.782
- 30. Classification of lung cancer from cytological images Teramoto et al. (2017) Data - FNA cytology/298 (images) Task – Classification into Adeno,squamous,small cell carcinoma Model – CNN Augmentation – Rotation,flip,inverting Performance - Overall classification acc. 71.1% Classification Accuracies trained by original and augmented images Lung Cancer
- 31. Sample images of correctly classified and misclassified carcinoma
- 33. Classification of lung cancer and mutation prediction from non- small cell lung cancer
- 35. Prostate Cancer • Currently, detection of tumor patterns,Gleason grading and combination of prominent grades into a Gleason score – critical in determining the clinical outcome of Prostate Cancer. • More recently,research teams have proposed use AI tech for automated analysis of prostate cancer. • Litjens et al. – convulational auto encoder for tumor detection in H&E stained biopsy specimen. • Bulten et al. – developed algorithm for automated segmentation of epithelial tissue in prostectomy slides using CNN
- 36. • Jimenez et al. – developed automated approach using patch selection and CNN to detect ROI in WSI. • This algorithm was able to differentiate between Gleason 3+4 and 4+3 slides with 75% accuracy. • Nagpal et al. – presented deep learning system for Gleason grading in WSI that quantifies the tumor morphology – opportunity for refinement of Gleason system itself.
- 38. • Ertosun et al. (2015) • Data – TCGA • Task – Glioblastoma multiforme vs Lower grade Glioma Further grading of LGG into Grade II or Grade III • Model – CNN • Performance - GBM/LGG decision acc. 96% LGG grade decision acc. 71% Brain cancer
- 39. • Mobadersany et al. (2018) • Data - TCGA-GBM & LGG/1,061 (samples) • Task - Survival analysis • Model – SCNN • Performance - C- index 0.754 • Mobadersany et al. (2018) • Data - TCGA-GBM & LGG/1,061 (samples) • Task - Survival analysis • Model – CNN • Performance - C-index 0.754
- 40. • Wu et al. (2018) • Data - 7,392 (images) • Task – Classification of ovarian tumor • Model - DCNN • Performance - Overall classification acc. 78.2% Ovarian Cancer
- 41. • Zhang et al. (2017) • Data - HEMLBC/1,978 Herlev/917 (images) • Task - Benign/cancerous cell • Model – CNN • Performance-Image level decision AUC 0.99 Cervical Cancer
- 42. • Xu et al. (2017) • Data - Red-blood cell/7,206 (patches) • Task - Classification of RBCs in sickle cell • Performance - Cell level classification acc. 87.5% Diagnosis of sickle cell anemia
- 44. IHC Application • IHC image analysis provides an accurate means for quantitatively estimating disease related protein expression. • Hence reducing inter and intra observer variation and improve scoring reproducibility. • Lejeune et al. –proposed algorithm for automated analysis for quantification of proteins for different nuclear (Ki-67,p53), cytoplasmic (TIA-1,CD-68) and membrane markers CD4,CD8,CD56,HLA-DR) • Chen et al. proposed CNN-based technique for automatic detection of immune cells in IHC images.
- 45. • Quantification of PD-L1 is made more difficult by the non-specific staining of areas other than tumor cell membranes,in particularmacrophages,lymphocytes,necrotic and stromal regions. • Humphries et al. proposed automated image based deep learning technique to quantify PD-L1 expression . PD-L1 imaging in lung cancer. Deep learning can be used to identify and distinguish positive | negative tumor cells and positive | negative inflammatory cells.
- 46. Genetic Mutation Prediction • Deep CNNs was used to predict whether or not SPOP was mutated in prostate cancer, given only the digital whole slide after standard H&E staining. • Kim et al. used deep convolutional neural networks to predict the presence of mutated BRAF or NRAS in melanoma histopathology images. • The findings from these studies suggest that deep learning models can assist pathologists in the detection of cancer subtype or gene mutations and therefore has the potential to become integrated into clinical decision making. SPOP - Speckle-Type POZ protein
- 47. Tumor Detection for Molecular Analysis • The increasing number of molecular tests for specific mutations in solid tumors has significantly improved our ability to identify new patient cohorts that can be selectively treated. • EGFR mutational analysis in lung cancer, KRAS in colorectal cancer and BRAF in melanoma all represent examples of mutational tests that are routinely performed on appropriate patients with these cancers. • Histopathological review of the H&E tissue section prior to molecular analysis is critical. • Clarity over the cellular content is critical to ensuring the quality of the molecular test. • So,the pathologist must routinely assess the % of tumor cells to again ensure that there is sufficient tumor DNA in the assay and that the background noise from non- tumor cells does not impact on the test result.
- 48. • Clarity over the cellular content is critical to ensuring the quality of the molecular test. • So,the pathologist must routinely assess the % of tumor cells to again ensure that there is sufficient tumor DNA in the assay. • The challenge is the interobserver variation in the assessment of percentage of tumor which range from between 20% and 80% increasing the risk of false negative molecular tests. • TissueMark developed by PathXL Ltd automatic analyse H&E tissue samples, identify the boundary of the tumor and precisely measure tissue cellularity and tumor cell content. • The algorithms have now been expanded to automatically identify tumor in colorectal, melanoma, breast, and prostate tissue section.
- 49. Automated analysis of cellular content in H&E using deep learning in TissueMark. Here tumor (red) and non-tumor cells (green) can be distinguished, annotated for visual inspection and counted to reach more precise qualititive measures of % tumor across entire whole slide H&E scans in lung, colon, melanoma, breast, and prostate tissue sections.
- 50. Augmented Reality Microscope (JENOPTIK) 1 2 3 4
- 51. Companies working for AI in Pathology Concentriq software
- 52. Advantages Quantification of Data Bringing every pathological investigations at one platform – A step toward indivisualised management of Patient. Recognizing new paraments and predicting survival rate, mutations. Time Automation Making data science accessible to pathologists
- 53. Challenges Pre-imaging technique (staining) Lack of labelled data • Most AI algorithms require a large set of good quality training images (labelled data) • Crowdsourcing data may be cheaper and quicker but has potential to introduce noise. Pervasive variability • The extreme polymorphism of cell make recognizing tissues by image algorithms extremely challenging
- 54. Dimensionality obstacle • WSI deals with gigapixel digital images (>50,000 by 50,000 pixels).However Deep neural network operates on (<350 by 350 pixels) • Deep nets with large input sizes would need much deeper topology and much larger no.of neurons making them even more difficult to train Affordability of required computational expenses • Pathology labs are already under immense financial pressure to adopt WSI • Asking for GPU as a prerequisite for training /using AI solution is consequently going to be financial limitation
- 55. Lack of transparency and interpretability • At present no established way to explain why a specific decision was made by a network while dealing with histopath scans. • And its unacceptable for clinicians as they need to justify the underlying reasons for a specific decision. Realism of artificial intelligence • Deploying AI tools in practice is difficult. • Pathologists buy-in to employ these tools will depend on three key features: Ease of use (uncomplicated preimaging demands and generalizable,scalable understandable output) Financial return on investment Trust(evidence of performance)
- 56. CONCLUSION • Via AI technology, Information about the state of our health can be extracted from tiny bioparticles and diseases like cancer can be detected and treated easily at the early stage . • AI will not only make an advanced connecting world but will be able to save more and more number of lives .
- 57. • AI will not replace pathologists. • But in future, the pathologists with training in AI will replace the pathologists without training in AI. Take Home Message
Editor's Notes
- #12: The word ‘deep’ in deep learning refers to the layered model architectures which are usually deeper than conventional learning models.