Deep learning platform
- 1. Covid-19 Tele Health Solution Deep Learning Platform By Bhagvan Kommadi
- 2. Agenda Introduction Telehealth Platform Deep Learning Architectural Components Implementation What’s Next ?
- 3. Introduction
- 5. 5 Genomics What is the functional form? How do we investigate these relationships? Can we take advantage of the exponential growth of genomic data? A, C, G, T 20,500 genes in human body
- 6. 6 Genomics : Genotype Phenotype = Genotype + Environment Traits Diseases Behaviors … Gene sequence SNP’s Expression data … Diet, smoking, drugs, … Insults and injuries Exposures … Perform Controlled Experiments? Unethical using human subjects!!! OK on rats.
- 7. 7 Phenotype and Environment Environment Questionnaires, • e.g., Nurses’ Health Study, Framingham Heart Study Monitoring • e.g., LDS hospital infectious disease monitors Phenotype Natural Experiments Clinical Data
- 8. 8 Vision: Integrate Informatics with Biology I2b2: I. Kohane, et al.
- 9. 9 Phenome – Genome Network Butte & Kohane, Nature Biotech 2006
- 10. Deep Learning Model Input Layer Hidden Layer Hidden Layer Hidden Layer Output Layer Mutation Collection Genetic Variation
- 11. Gene Regulation Genome Organization Mutation Effects Detecting Introgression Estimating historical recombination rates Identifying selective sweeps Estimating demography of population genetics Deep Learning Network Layers
- 12. mRNA alignments Mappings of DNA repeat Elements Gene Predictions Gene- Expression data Disease- Association data Deep Learning – Genome Models Relationships of genes to diseases Expression tracks • correlating genetic data with the tissue areas • linkage of a particular gene or sequence with body tissues
- 13. Type1 • HLA-DQA1 • HLA-DQB1 • HLA-DRB1 Type2 • TCF7L2 • SLC30A8 • HHEX • FTO • PPARG • KCNJ11 Gestational LADA MODY Double Types Diabetes– Types – Gene Variants
- 14. High triglycerides DNAm variations Genetic Expressions Blood Lipid Levels • high-density lipoprotein cholesterol • Low density • lipoprotein cholesterol • Triglycerides • Total cholesterol Diabetes– Genetic Expressions
- 15. Telehealth Platform Powered By AI Deep Learning Framework
- 17. Know me Be competent Care about me Improve / cure / prevent my problem What Patients Want
- 18. Tele Health Solution Web RTC based solution Deep Learning Based & Voice Historical Data : prediction and identifying patterns Neural Network Algorithms are used • Image Processing and Natural Language • Voice assistants conversing with the User Visualization Data Access Layer Personalizati on Recommendati on Engine Transaction Engine Wallet Curator Loyalty Mgmt CMS Integration OCR Learning Platform Podca st Vide o Service IOT Voice Event Hostin g Internet Data Sources Fact Checke r Payment Gateway eLearnin g Poll s Surve ys Gamificati on Cases Digital Pen IOT Gateway Insights Plugins Extension s Knowledge Engine
- 19. Doctor Community Platform Users Notifications Groups Content Moderation Recommendations Events Products Services Discussions Audio & Video Chats Polls Gamification News Blogs Employees Partners Vendors Customers Product Reviews Knowledge Graph Social Interactions Group Discussions Survey s Careers Courses Wallet
- 20. Voice Assistants
- 23. Process Flow Filters NLP API Web Mobile Desktop Queries Registry Data Sources Links Knowledge Base Updates 1) Diseases 2) Data Sources 3) Artifacts Top Down As Knowledge Domain Expert Bottom Up Registry & knowledge updates • Orphanet • Johns Hopkins • WHO • NORD
- 24. Architecture Service Layer Visualization Layer Data Access Layer Knowledge Update Internet Data Sources Search Manager Knowledge Base Manager axios cheerios node Disease Registry Disease Knowledge
- 26. Deep Learning HealthCare Data Analysis
- 27. input data output Simple Feed-Forward Network ... ... sigma-pi step function weights perceptron output Perceptron Image consisting of pixels and greyscale intensity ranges from 0 to 255 and weighted average is measured by the Perceptron
- 28. Iris setosa Iris versicolor Iris virginica Sepal /Petal 0.0 0.5 1.0 Iris-setosa Iris-versicolor Iris-virginica sepal length sepal width petal length petal width 4 2 1 0.28 0.01 10000 30 0.1835137273718 - 1.52185484488147 1.06085392071769 - 10.1057086709985 - 1.53328697751333 4.0131689222145 -1.6375908770170 6.66056158141261 input layer hidden layer output layer learning rate error limit max runs # training sets ihweights howeights 5.1 3.5 1.4 0.2 Iris-setosa 4.9 3.0 1.4 0.2 Iris-setosa 4.7 3.2 1.3 0.2 Iris-setosa 4.6 3.1 1.5 0.2 Iris-setosa 5.0 3.6 1.4 0.2 Iris-setosa : 7.0 3.2 4.7 1.4 Iris-versicolor 6.4 3.2 4.5 1.5 Iris-versicolor 6.9 3.1 4.9 1.5 Iris-versicolor 5.5 2.3 4.0 1.3 Iris-versicolor 6.5 2.8 4.6 1.5 Iris-versicolor : 6.3 3.3 6.0 2.5 Iris-virginica 5.8 2.7 5.1 1.9 Iris-virginica 7.1 3.0 5.9 2.1 Iris-virginica 6.3 2.9 5.6 1.8 Iris-virginica 6.5 3.0 5.8 2.2 Iris-virginica : 3 classes 50 samples each trained network specification 4-2-1 net Outputs Iris Flower Data Set
- 29. In real world, does this work ??
- 30. Train this layer first then this layer then this layer then this layer finally this layer Multi Layer NNs
- 31. Convolutional Neural Network Input Layer Hidden Layer Hidden Layer Output Layer Hidden Layer Hidden Layer
- 32. Image Recognition Image Pixels Pixel Value Color Depth Grey Scale Intensity CNN Layers Convolution Rectified linear unit (ReLu) Pooling Fully Connected Keras pre- trained CNNs Xception VGG16 VGG19 ResNet 50 InceptionV3 InceptionR esNetV2 Mobile Net Dense Net NASNet Mobile NetV2 Features Extraction
- 33. Atelectasis 192 Cardiomegaly 50 Consolidation 72 Edema 41 Effusion 203 Emphysema 42 Fibrosis 38 Hernia 5 Infiltration 503 Mass 99 No Finding 3044 Nodule 144 Pleural_Thickening 65 Pneumonia 14 Pneumothorax 114 Total Result 5606 Atelectasis Pneumothorax Lung Disease Analysis Gender : Male, Female Xray Position: Frontal , Back, Side
- 34. Lung Disease – Deep Learning Training Testing Validation Normal Infection by Virus Infection by Bacteria True Positive False Positive False Negative False Positive Actual P r e d i c t e d Precision = True Positive/Actual Results Recall = True Positive/Predicted Results Accuracy = (True Positive + True Negative)/Total Actual Results = True Positive + False Positive Predicted Results = True Positive + False Negative CNN Scale Shift Rotate Salt and Pepper noise Flip
- 35. Total params: 12,167,424 Trainable params: 12,167,424 Non-trainable params: 0 Precision: 0.734694 % Recall: 0.073022 % Fscore: 0.261248 % Convolutional Neural Network Results Training Data Bias = Average Predicted Value - Average Actual Value
- 36. Convolutional Neural Network - Basic
- 37. Convolutional Neural Network – Complete DataSet
- 39. Technical Architecture Service Layer Visualization Layer Data Access Layer Mobile & Web Python Django MySQL & MongoDB
- 40. AI Platform – IOT & Cloud Doctors, Polyclinics & Hospitals Pharmaceutical Cos. Insurance Cos. Patients Pharmacies Pathology & Diagnostics Educational institutes Associations & bodies Government(s) Allied Services IOT Devices Applications Services Delivery & Fulfilment Cloud Integration Monitoring Message Store HL7 FHIR XDR XDS
- 41. AI Platform – IOT Architecture Cloud IOT Devices Heart Rate Temperature Light Water Leaks Humidity Sound Proximity Accelerometer Smart Phone Orientation Lifetime Pendant Service Service Service Service G A T E W A Y API Gateway Patients Doctors
- 43. Demography Words General Appearance Sounds Physical Signs Smell Touch / Feel Xray Images Fluids Tissues What Has Happened Before” What I Have Seen Before” Patterns
- 44. NDHM Blueprint
- 46. White Coats Platform Network App Practice Plus Clinic Plus StaffPlus Patient Plus White Coats Platform
- 47. Registered Doctors* 3,00,000+ Locations 3,000+ Onboarded Doctors (App) 49,831 Doctors from Metro/Tier 1 & 2 ~68% Doctors from Tier 3 & 4 ~32% # of Specialties 125+ General Practitioners ~35% % of Drs : Specialists ~65% • Average Logins (DAU) Q1- 1052 ; Q2- 1090 (Growth 14%) • Average MAU (APP & Web) Q1- 19,868 ; Q2- 25,466 (Growth 28%) • Onboards Q1- ~4200 ; Q2- ~4000 ( - 4.7%) • Web users Q1- 34k ; Q2- 52k (Growth 53%) • App & Web Views Q1- 2.05L ; Q2- 2.8L (Growth 36%) • Social Interactions (APP) Q1- 1.27L ; Q2- 1.58L (Growth 25%) • Network Cases Q1- 800 ; Q2- 1000 (Growth 25%) Doctor Community - Solution
- 48. Q1 Q2 Growth WHITECOATS BEATS ARTICLES NETWORK CASES & POSTS TOTAL WHITECOATS BEATS ARTICLES NETWORK CASES & POSTS TOTAL # of Posts 114 267 381 138 318 456 20% # of App Views 15,506 22,843 38,349 23,427 24,668 48,095 25% # of Likes 524 769 1,293 769 826 1,595 23% # of Comments 72 352 424 154 436 590 39% # of Shares 991 1,144 2,135 1,322 1,095 2,417 13% App Promotion | Social Media | Facebook, Instagram, LinkedIn, Twitter, YouTube Web Promotion | Social Media | Facebook, Instagram - Promoting All About COVID-19, App content Alexa website ranking improved by 156,136 ranks 997,920 (June) | 841,784 (October) Email – App and Web Promotions SMS • Onboarding • Association users • Content Engagement • Weekly Digest, Event Promotions • App Upgrade Q1 Q2 Reach 3,908,901 2,143,186 Impressions 15,765,902 11,657,355 Clicks 46,636 42,968 Landing Page 22,687 29,428 Onboards 4,247 3,121 Verified 659 562 Q1 Q2 Web site visitors 37,194 57,962 Page Views 61,714 135,461 Bounce Rate 71.86% 61.44% Avg. Time Spent 01:21 Min 00:59 Min Q1 Q2 Delivered 1,301,311 2,219,838 Open 131,953 235,660 Click 16,187 30,145 Doctor Engagement Analytics
- 49. Doctors Onboarded 4,000+ (3000+in last 6 months) Clients/Channels 5+ Specialties 100+ Key clients/channels: • Pharma Companies) • Retail Doctors • Self-registered Doctors • Hospitals • Clinic Top Specialities: • GP/CP • Dermatologist • Diabetologist • Urologist • Cardiologist • Neurologist • Gastroenterologist • Paediatrician • OB/GY • ENT Specialist Patient Registrations 3.5 lakh+ Appointments 35,000+ EMR/Prescriptions 5,000+ Key channels: • Self-registered: 30,000+ • Dr or Staff registered: 2.5 lakhs + • Via Bulk Import: 1 lakh+ Key channels: • Clinic:30,000+ • Video: 5,000+ • Chat:1,000+ Formats: • E-Prescriptions • Handwritten Telehealth Solution
- 51. Genomic Applications Personalized Medicine Evidence Based Medicine Predictive Toxicology QSAR Chemo-Informatics What’s Next ?
- 52. Appendix
- 53. Semantic Analysis Grammatical relationships between words and phrases John hit the ball object subject modifier Dependency parse Positive Good Great Fantastic Excellent Friendly Awesome Spectacular Negative Bad Worse Issue Loss Awful Problem Bogus Full parse Entity Extraction, Fact Discovery, Intent & Sentiment Feeds Text Miner Blogs, News verb Named Entity Recognition
- 54. Voice Assistants Show us the patient’s report ! Search for patients generated report ! Search Results HTTP(S) Google DialogFlow Request Response
- 55. Speech Processing Speech Coding Synthesis Recognition Speaker Model Diarisation Speech Transcription Speech Type Speech Patterns Speaker Identity
- 56. Speaker Recognition Template Based word Recognition Continuous Speech Recognition Neural Network Based Recognition Analysis Feature Extraction Extraction Modelling Testing
- 57. Speech Types Isolated Words Connected Words Spontaneous Speech Simple Sentence Complex Sentence is John going to fall Although John was wealthy, he
- 58. Speaker Model Independent Dependent Large Group of People Particular Speaker
- 60. Speech Engine Environment Channel Speaker Style Gender Age Speech Speed
- 61. 61 Proprietary & Confidential: WhiteCoats is an offering of ValueMomentum Software Services Pvt. Ltd. Knowledge Base - Types Human Readable • Documents Formatted • FAQs • Manuals • How tos Machine Readable • Unstructured data • Reasoning • Explanation Knowledge Agents • Artifacts Learning (Neural Network) • Supervised Learning
- 62. Speech Analytics Emotions Locations Periods of Silence Non Speech
- 63. Deep Learning • Perceptron • Iris Flower Data set
- 64. • Machine Learning • Deep Learning
- 66. Machine Learning • Predictive Analytics • Discovery • Process • Predictive Model • Ensemble Methods
- 67. Discovery Smart &Automatic Data Profiling Built-in Transformation Capabilities Modern Visualisations Fully Integrated with the Most Robust Framework Distributions on the Market Explore and Profile all Datasets in the Data Lake
Editor's Notes
- #6: Peter Szolovits -- AI Applications in Medicine
- #7: Peter Szolovits -- AI Applications in Medicine
- #8: Peter Szolovits -- AI Applications in Medicine
- #9: Peter Szolovits -- AI Applications in Medicine
- #10: Peter Szolovits -- AI Applications in Medicine
- #17: Book consultations online (Video/Audio) Get medicines delivered Book diagnosis tests Set reminders for medicines Ask questions to experts Add and save medical records View appointments and diagnosis history View chat history and order history Add and view family members View and update profile Read articles and share them
- #44: +…”What Has Happened Before” +… “What I Have Seen Before”
- #60: Diarisation is partitioning an audio stream into homogeneous segments based on the speaker identity. Speech synthesis is a output form of speech e.g.,text-to-speech (TTS).