Clinical Decision Making with Machine Learning
2 likes1,133 views
Clinical decision making with Machine Learning In this document, the author discusses how machine learning can be applied to optimize clinical decision making in in vitro fertilization (IVF). Specifically, the author presents how their clinic collects vast amounts of IVF data and is working to build predictive models using machine learning techniques to personalize treatment protocols and embryo selection. Key areas discussed include using time-lapse imaging, preimplantation genetic testing, and endometrial receptivity analysis to select the best embryos, as well as collecting extensive clinical factors on over 918 single embryo transfer cycles to build a model that can predict IVF outcome with 73% accuracy.
![Building the model to predict IVF outcome
(FETs only)
Relative
Importance
Feature Description
0.95784
403_NumCatTE_Prior full
term_Prior pre-term_TE_0
Out-of-fold mean of the response grouped by: ['Prior full term',
'Prior pre-term', 'TE'] using 5 folds (numeric columns are
bucketed into 25 equally populated bins)
0.55907
164_CV_TE_# EXT
CULTURE_FACNAME_LUPRON_
PGD.1_Retrieval MD_Retrieval
technician_Thawing
technician_0
Out-of-fold mean of the response grouped by: ['# EXT
CULTURE', 'FACNAME', 'LUPRON', 'PGD.1', 'Retrieval MD',
'Retrieval technician', 'Thawing technician'] using 5 folds
0.35233 217_BIOCHEMICAL BIOCHEMICAL (original)](https://image.slidesharecdn.com/h2opresbarash2-180303010205/85/Clinical-Decision-Making-with-Machine-Learning-55-320.jpg)
Clinical Decision Making with Machine Learning
- 1. Clinical decision making with Machine Learning Oleksii Barash, Ph.D. Reproductive Science Center of San Francisco Bay Area
- 2. Disclosure We have no financial relationship with any commercial interest related to the content of this activity
- 3. Reproductive Science Center of the SF Bay Area • Founded in 1983 • In top 30 largest IVF (In Vitro Fertilization) clinics in USA* • In top 20 clinics with the best clinical outcomes* • Over 2000 treatment cycles (fresh and frozen) in 2017 * - CDC Report 2015
- 4. What is infertility? WHO - Infertility definitions and terminology • Failure to conceive within 12 months of regular unprotected intercourse. • Primary or secondary. • 84% of couples will conceive within 1 year and 92% within 2 years.
- 5. Scope of the problem • Infertility affects 12% of the reproductive age population in the US (≈12 million people) • Infertility affects men and women equally • More than 50% of infertility patients will have a baby with treatment • Over 1.5M IVF cycles per year worldwide (≈ 200,000 in USA) in 2014 • Cost of one IVF cycle in US: 10K – 100K
- 6. Global fertility Market Equity Research Reports, 2012 Key growth drivers: 1. Aging and Infertility 2. Increasing prevalence of Obesity 3. Cultural shifts (“Celebrities” and LGBTQ)
- 7. Unreasonable expectations… • 59% of childless women aged 35‐39 still planned to have a baby • 30% aged 40‐45 did too! (Sobotka, Austrian survey data) • 58% said they wanted 2 children (aged 21‐23) • Only 36% had achieved that by age 36‐38 (Smallwood and Jeffries,UK Population Trends)
- 10. IVF is essentially manufacturing • Complex multidimensional process; • Constant intake flow of the patients; • Cutting edge labor and equipment; • Hundreds of contributing factors (Lab + Clinical); • Every patient is unique – limited standardization Ultimate goal – single healthy baby
- 12. IVF produces a lot of data? • Main shareholders are open to cutting edge technologies • Wide Electronic Medical Records adoption; • IoT devices – sensors, incubators, microscopes, lasers • Morpho-kinetics (time-lapse) • Preimplantation Genetic Testing • “Omics”
- 13. Transforming data into knowledge • Increasing number of publications • Retrospective and small • Rare RCTs
- 14. Evidence based medicine Conscientious, explicit and judicious use of current best evidence in making decisions about the care of an individual patient.* * - Sackett. BMJ 1996;312:311-2
- 15. Meta-Analysis Fertility and Sterility 2010; 94:936-945 • Small number of samples • Diverse experimental conditions
- 16. Personalized decisions to be made in each IVF cycle • Hormonal Stimulation protocol / dosage / duration • Lutheal support • How many embryos to transfer (1, 2 or 3) • Embryo selection for the transfer (morphological and genetic) • Financial products (risk sharing programs, money back)
- 17. Do You Know Your Embryo Biology?
- 18. Time-lapse and Machine learning
- 19. Embryo selection for the transfer • From 1 to 30+ embryos per IVF cycle • Many morphological and kinetic features per embryo • Critical choice – no second chance
- 20. Traditional embryo evaluation M. Montag, 2014
- 21. Time-lapse monitoring M. Montag, 2014
- 22. Non-invasive imaging and predictions
- 23. EEVA (Early Embryo Viability Assessment) • Xtend algorithm: – over 1,000 combinations of potential parameters – includes egg age, cell count and Post P3 analysis – which measures cell activity after the four cell stage – Post P3 is the result of a proprietary analysis based on 74 computer-based attributes that are combined into one parameter – each embryo gets a developmental potential score ranging from 1 (highest) to 5 (lowest). – 84% specificity vs 52% by traditional assessment – The odds ratio of predicting blastocyst formation is 2.57 vs 1.67 by traditional assessment
- 24. EEVA (Early Embryo Viability Assessment)
- 29. Oocyte aneuploidy and maternal age Handyside, 2015 • All primary oocytes are formed before baby- girl is born.
- 30. Preimplantation Genetic Testing Handyside, 2015 DNA sequencing DNA flow cell
- 31. Preimplantation Genetic screening National Human Genome Research Institute, 2014 • Log scale!
- 32. Preimplantation Genetic Testing National Human Genome Research Institute, 2017 • Log scale!
- 33. Single Nucleotide Polymorphism (SNP) algorithm • 300,000 probs per embryo • Per chromosome confidence • Highly accurate and comprehensive results • Parental genomic information • Cumulative distribution function (cdf) curves
- 34. Cumulative live birth rate after SET, PGS, N=1024 # Cycles Live births Total ETs 1-l/n S(t) 1 178 313 0.43131 0.56869 2 22 59 0.627119 0.72952 3 7 15 0.533333 0.85574 4 1 2 0.5 0.92787 5 1 1 0 1 Presented by RSC team at ASRM 2016
- 35. Gene expression, stage & multinucleation
- 36. ML-based solutions for IVF
- 37. Univfy Univfy algorithm: • Takes patient data • Predictive model based on 13,000 IVF cycles; • Chances for positive outcome • Chances of twins if 2 embryos were transferred
- 38. Celmatix Celmatix algorithm: • Incorporated in our EMR (ARTworks) • Software as a service (SaaS model) • Data analytics platform to help optimize patient management and counseling
- 39. Celmatix - Fertilome Celmatix algorithm: • 25,000 peer-reviewed studies • 1,713 genes • 427 variant/diagnosis combinations • 201 gene-diagnosis combinations • 32 target genes in the kit
- 40. Endometrial Receptivity Analysis (ERA) by Igenomix Patented in 2009: PCT/ES 2009/000386 Customized microarray (238 genes) Bioinformatic analysis of data obtained by the customized microarray Classification and prediction from gene expression.
- 41. Endometrial Receptivity Analysis (ERA) Receptive Model Classifies the Molecular Receptivity Status of the Endometrium Post-ReceptivePre-Receptive
- 42. 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% 2010 2011 2012 2013 2014 2015 2016 %ofcycles ETx1 ETx2 ETx3 ETx4 ETx5 ～ Average age – 36.0 ± 5.5 y.o. ～ 39.3% of all patients are over 38 y.o. SET rate in non-PGT cycles (2010-2016), fresh D5 ET, N=3925
- 43. Preimplantation Genetic Testing (PGT) at RSC ～ SET frequency in PGS IVF cycles (average age – 37.5 ± 4.29 y.o. ) – 89.9% FISH SNP – aCGH - NGS 661 1387 4 735 0 200 400 600 800 1000 1200 1400 0 200 400 600 800 1000 1200 1400 1600 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 2014 2015 2016 2017 NumberofIVFcycles Total volume PGT cases 78
- 44. Live birth rate Maternal age Number of embryos for biopsy Morphology of the embryos SET vs eSET D5 vs D6 Biopsy Total gonadotropin dosage Number of previous failed cycles Number of normal embryos per cycle Number of eggs Euploidy rate Presented by RSC team: ASRM 2016, 2015, 2014; ESHRE 2015, 2016; PCRS 2014, 2015, 2016; PGDIS 2015, 2017 Factors affecting PGT outcomes
- 45. Live birth rate Embryo _Age Blastula tion_rat e Donor_ eggs Euploid y_rate Number _of_nor mal d5_to_t otal_rat io Total_d ay_5_bx Total_d ay_6_bx Total_fo r_biosy Bx_Day Emb_Ex pansion ICM TE Gender Best_E mbryo_ For_ET Elective _SET Cycle_n umber Number _of_Foll icles Zygotes Fert_rat e Unfert M2 M1 GV ATR Multi_P N PN_1 Degene rated Cleaved Cleavag e_rate Number _ext_cu ltureGood_e xt_cultu reNumber _to_blNumber _CryoGood_d 3_rateTVA_M D Number _of_tar nsfers_t o_deliv ery Semen_ Source Fresh_F rosen_s p BMI PATIEN TTYPET EXT NO_OF _DAYS SUMSTI M ASPIRA TED_O OCYTES HCG_D RUG TOTAL2 PN GRAVID ITY PREM TERM SAB BIOCHE MICAL LIFETIM E_SMO KED PRIORIV F PRIORF ET PRIORI UI HEIGHT WEIGHT PRIMAR YDIAGN OSIS SEMENS OURCE FSHLEV EL NEARES T_AMH MED1 Peak_E 2 TOTALI US FOLLICL ES_BIG GER_TH AN_14 ASPIRA TED_O OCYTES NO_FR OZEN NO_VIT INITIAL CONSUL T_PREM INITIAL CONSUL T_GRAV IDITY INITIAL CONSUL T_SAB INITIAL CONSUL T_TERM INITIAL CONSUL T_BIOC HEMICA L Stim protoco l Factors affecting PGT outcomes More factors? Bias? Reproducibility of the results?
- 46. Factors affecting PGT outcomes What if we can evaluate ALL available factors?
- 47. What if we can assess ALL available factors? 20 factors: 202 = 400 plots 381 factors 3812 = 145,161 plots 20 x 20 Machine Learning
- 48. Algorithm Timeframe: Jan 2013 – Jul 2017 Retrospective analysis Number of PGS transfers: 918 Average age: 35.6 ± 4.8 ONLY Single embryo transfers Machine learning methods: • GLM (Generalized Linear Models) • RPART (Classification and Regression Trees) • GBM (Generalized Boosted Regression Models) IVF lab Embryo_Age Blastulation_rate Donor_eggs Euploidy_rate Number_of_normal d5_to_total_ratio Total_day_5_bx Total_day_6_bx Total_for_biopsy Bx_Day Embryo_Morphology Expansion ICM TE Gender Clinical_Outcome BEST_ EMBRYO_FOR_ET ELECTIVE_SET Number_of_tarnsfers_to_delivery Biopsy tech CYCLE # PEAK E2 TVA MD TVA TECH # Follicles >12 mm # EGGS # INSEM # 2PN % FERT # UNFERT #M2 or mature # INT # IMM # ATR # > 2PN # 1PN # DEG FERT CK TECH ICSI TECH SEMEN SOURCE FRESH/FROZEN SP CLEAVED % CLEAVED HATCH TECH # EXT CULTURE # GOOD EXT CULT # TO BLAST # CRYO % OF GOOD QUALITY EMBRYOS … clinical BMI PRIMARY_DX PATIENTTYPETEXT LUPRON STIM GNRHA MED1 SUMSTIM TRANSFER_DATE HCG_DRUG GRAVIDITY PREM TERM SAB BIOCHEMICAL PATIENTRACE LIFETIME_SMOKED SMOKING_FREQ PRIORIVF PRIORFET PRIORIUI HEIGHT WEIGHT STIMPROTOCOL LUPRONPROTOCOL PRIMARYDIAGNOSIS SECONDARYDIAGNOSIS TERTIARYDIAGNOSIS SEMENSOURCE PATIENTTYPE FSHLEVEL E2LEVEL NEAREST_AMH AFC MED1 MED2 MED3 MED4 MAX_E2 TOTALIUS FERT_METHOD_ICSI FERT_METHOD_IVF INITIALCONSULT_PREM INITIALCONSULT_GRAVIDITY INITIALCONSULT_SAB INITIALCONSULT_TERM INITIALCONSULT_BIOCHEMICAL Stim protocol … 381 variables per SET:
- 49. Lab factors, 918 SETs Pregnant, %Non-Pregnant, % % of total SETs Yes No
- 50. Lab + Clinical, 918 SETs
- 51. Relevant feature selection algorithm* (Lab factors) *Number of CART trees = 100
- 52. Relevant feature selection algorithm* (Lab + Clinical) *Number of CART trees = 100
- 53. Building the model to predict IVF outcome Only weak predictors are present Relatively small sample size (10K) A lot of features (>300) Accuracy of predictions = 0.73 AUC = 0.76 (Sensitivity/specificity balance)
- 54. Building the model to predict IVF outcome (PGT only) • Benchmark AUC – Starting point • Feature engineering • Feature importance • Feature transformations • Non-important features • Model interpretation
- 55. Building the model to predict IVF outcome (FETs only) Relative Importance Feature Description 0.95784 403_NumCatTE_Prior full term_Prior pre-term_TE_0 Out-of-fold mean of the response grouped by: ['Prior full term', 'Prior pre-term', 'TE'] using 5 folds (numeric columns are bucketed into 25 equally populated bins) 0.55907 164_CV_TE_# EXT CULTURE_FACNAME_LUPRON_ PGD.1_Retrieval MD_Retrieval technician_Thawing technician_0 Out-of-fold mean of the response grouped by: ['# EXT CULTURE', 'FACNAME', 'LUPRON', 'PGD.1', 'Retrieval MD', 'Retrieval technician', 'Thawing technician'] using 5 folds 0.35233 217_BIOCHEMICAL BIOCHEMICAL (original)
- 56. Ongoing PR after SET with different blastocyst morphology (918 SETs) Blastocyst morphology AA AB BA BB B-/-B p-Value Total SETs 266 292 33 232 95 n/a Positive hCG 222 240 26 178 61 n/a Negative hCG 44 52 7 54 34 n/a Biochemical 25 23 1 36 16 n/a Miscarriages 18 17 3 14 6 n/a Ongoing PR per ET, % 67.3 68.5 66.7 55.2 41.1 p<0.05 Birth outcomes (2013-2015) 107 135 8 117 61 n/a Live births 66 82 4 56 26 n/a Live birth rate, % 61.7 60.7 50.0 47.9 42.6 p<0.05 http://www.ivfbigdata.com/pgt-calculator/
- 57. eSET FUTURE SET vs DET in PGS cycles (2013-2016) ETx1 ETx2 P-value Total FETs 569 89 Positive HCG 442 78 Negative HCG 127 11 Ongoing pregnancies 335 66 Ongoing PR, % 58.9% 74.2% p<0.00599 Live birth rate,% 53.5% 71.6% p<0.00523 Twins 3 33 (1 triplet) Twin rate 0.9% 50.0% p<0.00001 Presented by RSC team at ASRM 2016
- 58. The 5 Steps Towards Evidence Based Practice 1. Ask the right clinical question: Formulate a searchable question 2. Collect the most relevant publications: Efficient Literature Searching Select the appropriate & relevant studies 3. Critically appraise and synthesize the evidence. 4. Integrate best evidence with personal clinical expertise, patient preferences and values: Applying the result to your clinical practice and patient. 5. Evaluate the practice decision or change: Evaluating the outcomes of the applied evidence in your practice or patient.
- 59. The 5 Steps Towards Evidence Based Practice 1. Ask the right clinical question: Formulate a searchable question 2. Collect the most relevant DATA: Efficient Literature Searching Select the appropriate & relevant studies 3. Critically appraise and synthesize the evidence. 4. Integrate best evidence with personal clinical expertise, patient preferences and values: Applying the result to your clinical practice and patient. 5. Evaluate the practice decision or change: Evaluating the outcomes of the applied evidence in your practice or patient.
- 60. The current problem with the models: A vs B
- 61. Conclusion 1. Machine learning is not yet widely used in clinical practice 2. Augmented decision making with machine learning 3. Auto ML for rapid experimentation knowledge discovery
- 62. Thank you! Lab: K. A. Ivani, Ph.D. O. O. Barash, Ph.D. N. Huen S. C. Lefko C. MacKenzie J. Ciolkosz E. Homen E. Jaramillo MDs: L. N. Weckstein S. P. Willman M. R. Hinckley D. S. Wachs E. M. Rosenbluth S. P. Reid M. V. Homer E. I. Lewis