Fallacies indrayan
- 1. STATISTICAL FALLACIES AND ERRORS IN MEDICAL RESEARCH A. Indrayan PhD(OhioState), FAMS, FRSS, FASc Delhi
- 2. CONSIDER Almost everybody has more legs than the world average
- 3. Head in an oven and feet in a freezer, and the person is comfortable ON AVERAGE???
- 4. Most people die in bed —avoid bed to prolong life!!!
- 5. ALSO • Bikini • Lies • Curse of Kelvin • Ghost of Gauss
- 6. I. BIASED SAMPLE • Survivors (lung functions and old age) • Volunteers (trials) • Clinic subjects (severe, well-to-do) • Publication bias (no other source)
- 7. II. INADEQUATE SIZE OF SAMPLE • Freiman et al. studied 71 negative RCTs – sample size too small to detect 25% improvement; Dimmick et al. reported similar findings for surgical trials • Small sample has high SE (not a problem) but very likely to be not representative • More important for rare outcomes, multivariable situations • Problems with calculation of sample size
- 8. III. INCOMPARABLE GROUPS • Women on oral contraceptives were observed for thromboembolism but not those on mechanical devices • Difference in composition • Peptic ulcer in blood groups O, A, B and AB 45%, 35%, 15% and 5%. Group Number of Subjects Number Responded Response Rate (%) I. Treatment group 40 32 80.0 Mild 30 26 86.7 Severe 10 6 60.0 II. Control group 40 25 62.5 Mild 8 7 87.5 Severe 32 18 56.2
- 9. IV. MIXING OF GROUPS
- 10. BMI Hb NOW ADD AN OUTLIER
- 11. BMI Hb Lower SE class Upper SE class SCHOOL-1 SCHOOL-2
- 12. 100 200 300 400 Serum cholesterol level (mg/dL) n=500 Normocholestremics n=100 Hypercholestremics Composite curve
- 13. V. IGNORING REALITY Looking for linearity Overlooking assumptions – uniformity of variance (BP, BMI), n for logistic Area under the concentration curve Anamolous person-years
- 14. VI. CHOICE OF ANALYSIS • Mean or proportion (Hb) • Forgetting baseline values - Hb • Interpretation of relative risk (from 10% to 50% is five times and from 1% to 5% is also five times)
- 15. VII. MISUSE OF STATISTICAL PACKAGES • Over-analysis (multiple comparisons on the same data) • Data dredging (reanalyze after deleting inconvenient values) • Torture the data until they confess • Quantitative analysis of codes - scores
- 16. VIII. ERRORS IN PRESENTATION Misuse of percentage 2 out of 5 – 40% and 3 out of 5 – 60% A gain of 20%! If waiting time for a surgery reduced from 6 months to 1 month due to administrative changes, is the reduction 500%, or 83%? If efficacy of a regimen is 80% against 60% of the existing, is the gain 20% or one-third?
- 17. Misuse of means • Head, oven, freezer… n, SD • ±SD, ±SE (± is valid at all?) • How about ±2SD (some use even SE)? • Outlier values
- 18. Unnecessary decimals • P=0.0000075 • Survival time 3.7534 years • In regression b=1.07 if birth weight is in kg same as 0.00107 if it is in g •Mean, SD, one extra decimal
- 19. Misuse of graphs • n is disregarded (1 out of 2 is 50% so is 45 out of 90 – both shown same way) • SD rarely depicted (±SD or ±2SD) • Statistical significance not shown
- 20. b 0 50 100 150 0 10 20 30 40 50 60 70 Age (years) Average systol ic BP (mmH g) a 115 120 125 130 135 140 145 0 10 20 30 40 50 60 70 Age (years) Average systol ic BP (mmH g)
- 21. b 0 50 100 150 0 10 20 30 40 50 60 70 Age (years) Average systol ic BP (mmH g) a 115 120 125 130 135 140 145 0 10 20 30 40 50 60 70 Age (years) Average systol ic BP (mmH g)
- 23. IX. MISINTERPRETATION Misuse of P-values • “Magic” threshold 0.05 (P more for quasi- random sample, nonGaussian distn.) • One-tail or two-tail • Dramatic P-values (P<0.000,000,000,01) (many packages stop at 3 decimals) • P-values for nonrandom samples • Multiple P-values • Too much emphasis on P-values - must be accompanied by biological explanation and plausibility
- 24. Correlation vs. cause-effect • Positive correlation between visual acuity and vital capacity in people of age > 50 years • Negative correlation between birth rate and CVD mortality in India since 1950 • Correlation between depression and risk of lung cancer due to smoking (seemingly nonsense correlation can generate a useful hypothesis)
- 25. Correlation vs. agreement • One sphygmo gives 4mmHg higher every time (bubble) than the other. Agreement is poor but r=1!!! • Comparison of test values with gold standard, and comparison of two tests when both can be in error—efficacy vs. agreement
- 26. Sundry issues • Medical significance vs. statistical significance • Missing out medically important gains— negative trials • Interpretation of SE (p) p = 0.03 and SE(p) = 0.022 vs. p = 0.40 and SE (p) = 0.044
- 27. Univariate vs. multivariate • Effect of maternal smoking on “growth”. Growth is multivariate (weight, length and head circumference). Individual variables can be significant but not when considered together. • If each healthy measurement has chance 0.05 being declared abnormal, what is the chance of doing so by 5 measurements together?
- 28. X. FINAL COMMENT • If a child cuts the finger, blame the knife? • Unintentional – misuse; Intentional – abuse • Dangerous tools in the hands of an inexpert • Correct statistics tell the truth, the whole truth, and nothing but the truth • Perceive Biostatistics as the science of managing medical uncertainties
- 29. There is no substitute for common sense—do not replace it with “evidence”—empiricism vs. rationalism