Stats 101: A Blueprint to Mastering Numeracy & Avoiding Apophenia

Today we are honored and privileged to have one of the premier minds in the field of critical care epidemiology, Dr. Burton Lee. In addition to his role as program director of the Pulmonary Critical Care & Emergency Medicine Critical Care Fellowship at the MedStar Washington Hospital Center, Dr. Lee spends a significant amount of time transforming the most complex statistical analysis techniques into such simplistic terms that even a novice can fully comprehend. Today we learn the tips and tricks to tackle numeracy and the α-error, 2 topics that you THINK you know, but do you really get it?!?

Podcast: Play in new window | Download

Subscribe: Apple Podcasts | RSS

Pearls: 

• Numeracy: mathematical competence, ie: applying simple concepts of mathematics to reason
• Apophenia: seeing patterns in randomness/meaningless data (example: pictures in the clouds)
  • Type I error or α-error: conclude a “pattern” or “result” when data is truly random 
    
  • Type II error or β-error: incorrectly conclude no pattern exists
• In order to be an astute reader of the medical literature you must understand the pitfalls of four commonly used statistical methods:
  • Multiple Testing: with multiple testing of the same concept, eventually you will get a positive study (by chance alone)
    
  • • Will see an eventual elimination of α-error
      • ex: Wenzel. NEJM 2004; 350:105-13.
  • Outcome changes: fitting the outcome of a study to a more publishable, “positive outcome”
    • Lessens adherence to α-error and hurts the overall power of the study
      • JAMA 2009;302:977-84
    • Can be remedied with preregistered outcomes: http://clinicaltrials.gov/
  • Publication bias: publication of only studies that support a desired outcome
    • Turner. NEJM 2008;358:252-60
    • The only solution is to report ALL clinical trials, again reliance on http://clinicaltrials.gov
  • Subgroup analysis: making conclusions by post-hoc analysis of randomly chosen subgroups
    • ISIS-2. Lancet 1988;ii:349-60
    • Remedied with pre-specification of subgroups prior to experimentation (not meant to replace hypothesis generation)
    • Criticize existing subgroup analysis by 2 methods:
      1. Draw a vertical line through the “all” group, if it can hit all the subgroups, there is no significance!
      2. Just check the p-value of heterogeneity (interaction)
         

De Backer. NEJM 2010;362:779-789