Normal Curves: Sexy Science, Serious Statistics
- Folgen
- 47
- Erscheint
- alle zwei Wochen
- Letzte Folge
- 2. Nov. 2026
Über den Podcast
Normal Curves is a podcast about sexy science & serious statistics. Ever try to make sense of a scientific study and the numbers behind it? Listen in to a lively conversation between two stats-savvy friends who break it all down with humor and clarity. Professors Regina Nuzzo of Gallaudet University and Kristin Sainani of Stanford University discuss academic papers journal club-style — except with more fun, less jargon, and some irreverent, PG-13 content sprinkled in. Join Kristin and Regina as they dissect the data, challenge the claims, and arm you with tools to assess scientific studies on your own.
Mitwirkende
- Kristin SainaniModeration
- Regina NuzzoModeration
Folgen 47
Researching Honesty Part 2: Can innocent mistakes explain the data?
Does cheating make you more creative? A provocative study from Harvard Business School suggested that it does. But data detectives uncovered suspicious patterns in the results, and a Harvard investigation found …
Shownotes
Does cheating make you more creative? A provocative study from Harvard Business School suggested that it does. But data detectives uncovered suspicious patterns in the results, and a Harvard investigation found even more troubling changes in the data. In Part 2 of our investigation into the Francesca Gino research misconduct case, we dig into the now-retracted “Evil Genius” study and the suspicious changes that made its results look better, then turn to Harvard law professor Lawrence Lessig's defense of Gino: Could innocent mistakes by “an army of research assistants” explain what happened? Along the way, we explore the trap of circular reasoning in statistics, why good data management matters, and how much responsibility researchers bear for their data even when it's out of their hands.
Statistical topics
- Statistical sleuthing
- Data forensics
- Data integrity
- Data cleaning
- Data management
- Reproducibility
- Statistical significance
- Spreadsheets
- Predictor variables
- Outcome variables
- Circular reasoning
- Research misconduct
Methodologic Morals
- “You cannot define your predictor variable using your outcome variable.”
- “If it takes an army of research assistants to clean your data, you're probably doing something wrong and maybe you need a statistician.”
- “Being too smart to commit stupid fraud is not a defense. Smart people do stupid things all the time.”
- “The more hoops your explanation has to jump through, the less convincing it becomes.”
References
- Gino F, Wiltermuth SS. Evil genius? How dishonesty can lead to greater creativity. Psychol Sci. 2014;25(4):973-981. Retracted. doi:10.1177/0956797614520714
- Retraction notice: Evil genius? How dishonesty can lead to greater creativity. Psychol Sci. 2023;34(8):947. doi:10.1177/09567976231187595
- Simonsohn U, Simmons JP, Nelson LD. Data falsificada (part 3): “The cheaters are out of order.” Data Colada. June 23, 2023. https://datacolada.org/111
- Simonsohn U, Simmons JP, Nelson LD. Exhibits 3, 4, and 5. Data Colada. September 16, 2023. https://datacolada.org/114
- Simmons JP. Harvard's Gino report reveals how a dataset was altered. Data Colada. July 9, 2024. https://datacolada.org/118
- Lessig L. The Gino Case. Website and podcast series presenting arguments challenging Harvard's research misconduct findings. https://theginocase.info/
- Simonsohn U, Simmons JP, Nelson LD. Data Falsificada. ResearchBox. Research data, analysis code, and supporting materials accompanying the Data Colada investigations. https://researchbox.org/1630.34
- Harvard Business School. Investigative report concerning allegations of research misconduct involving Francesca Gino. Harvard Business School Research Integrity Investigation Report. 2023. Accessed October 18, 2026. https://datacolada.org/wp-content/uploads/Harvard-Report-on-Gino.pdf
Kristin and Regina’s online courses:
Demystifying Data: A Modern Approach to Statistical Understanding
Clinical Trials: Design, Strategy, and Analysis
Medical Statistics Certificate Program
Epidemiology and Clinical Research Graduate Certificate Program
Program that Kristin teaches in::
Epidemiology and Clinical Research Graduate Certificate Program
Kristin’s manuscript writing course:
https://instats.org/seminar/from-data-to-discussion-writing-a-scient
Find us on:
Kristin - LinkedIn & Twitter/X
Regina - LinkedIn & ReginaNuzzo.com
- (00:00) - Intro
- (06:50) - The Claim: Superstars and Fraud
- (08:09) - Lessig’s Army of Research Assistants
- (22:15) - Sorted Data and 13 Suspicious Values
- (27:32) - Harvard’s Recovered Files from Gino’s Laptop
- (30:30) - Twelve People Reclassified as Cheaters
- (34:01) - Four RAT Totals That Don’t Add Up
- (45:25) - Lessig’s Research Assistant Theory
- (49:03) - Circular Logic and the Googling Defense
- (01:04:39) - Where the Evidence Leaves Us
- (01:07:18) - Rating the Claim
- (01:09:38) - Methodological Morals
Kapitel (12)
Researching Honesty: Can you trust the data? Part 1
A famous Harvard researcher. A blockbuster finding about honesty. Suspicious data, failed replications, and a $25 million lawsuit. Welcome to one of the strangest research misconduct scandals in recent memory …
Shownotes
A famous Harvard researcher. A blockbuster finding about honesty. Suspicious data, failed replications, and a $25 million lawsuit. Welcome to one of the strangest research misconduct scandals in recent memory. In the first of two episodes about the Francesca Gino case, we investigate a famous experiment claiming that signing an honesty pledge at the top of a form makes people less likely to cheat. We follow the trail from a failed replication to some extraordinary detective work involving an Excel spreadsheet—and conduct a little statistical sleuthing of our own. Along the way, we discover how suspicious changes to a handful of observations can alter the story a scientific study tells.
Statistical topics
- Statistical sleuthing
- Data integrity
- Data management
- Replication
- Sensitivity analysis
- Effect sizes
- Statistical significance
- Reproducibility
- Spreadsheets
- Conceptual replication
- Direct replication
- CalcChain
Methodologic Morals
- “There's cleaning your data, and then there's laundering your data.”
- “Good data record keeping is still sexy. Version control, baby.”
References
- Shu LL, Mazar N, Gino F, Ariely D, Bazerman MH. Signing at the beginning makes ethics salient and decreases dishonest self-reports in comparison to signing at the end. Proc Natl Acad Sci U S A. 2012;109(38):15197-15200. Retracted. doi:https://doi.org/10.1073/pnas.1209746109
- Kristal AS, Whillans AV, Bazerman MH, Gino F, Shu LL, Mazar N, Ariely D. Signing at the beginning versus at the end does not decrease dishonesty. Proc Natl Acad Sci U S A. 2020;117(13):7103-7107. doi:https://doi.org/10.1073/pnas.1911695117
- Retraction for Shu et al., Signing at the beginning makes ethics salient and decreases dishonest self-reports in comparison to signing at the end. Proc Natl Acad Sci U S A. 2021;118(38):e2115397118. doi:https://doi.org/10.1073/pnas.2115397118
Data Colada investigations- Simonsohn U, Simmons JP, Nelson LD. Evidence of fraud in an influential field experiment about dishonesty. Data Colada. August 17, 2021. https://datacolada.org/98
- Simonsohn U, Simmons JP, Nelson LD. Data falsificada (part 1): “Clusterfake.” Data Colada. June 17, 2023. https://datacolada.org/109
- Simonsohn U, Simmons JP, Nelson LD. Data falsificada (part 2): “My class year is Harvard.” Data Colada. June 20, 2023. https://datacolada.org/110
- Simonsohn U, Simmons JP, Nelson LD. Data falsificada (part 3): “The cheaters are out of order.” Data Colada. June 23, 2023. https://datacolada.org/111
- Simonsohn U, Simmons JP, Nelson LD. Data falsificada (part 4): “Forgetting the words.” Data Colada. June 30, 2023. https://datacolada.org/112
- Simonsohn U, Simmons JP, Nelson LD. Exhibits 3, 4, and 5. Data Colada. September 16, 2023. https://datacolada.org/114
- Simmons JP. Harvard's Gino report reveals how a dataset was altered. Data Colada. July 9, 2024. https://datacolada.org/118
Additional resources- Lessig L. The Gino Case. Website and podcast series presenting arguments challenging Harvard's research misconduct findings. https://theginocase.info/
- Gino v Harvard University. CourtListener. Federal court docket and publicly available filings. https://www.courtlistener.com/docket/67659904/gino-v-harvard-university/
- Simonsohn U, Simmons JP, Nelson LD. Data Falsificada. ResearchBox. Research data, analysis code, and supporting materials accompanying the Data Colada investigations. https://researchbox.org/1630.34
- Kristal AS, et al. Research materials and data accompanying Signing at the beginning versus at the end does not decrease dishonesty. Open Science Framework. https://osf.io/4b7mu/overview
- Correction for Kristal et al., Signing at the beginning versus at the end does not decrease dishonesty. Proc Natl Acad Sci U S A. 2026;123(27):e2610441123. doi:https://doi.org/10.1073/pnas.2610441123
- Harvard Business School. Investigative report concerning allegations of research misconduct involving Francesca Gino. Harvard Business School Research Integrity Investigation Report. 2023. Accessed October 18, 2026. https://datacolada.org/wp-content/uploads/Harvard-Report-on-Gino.pdf
Kristin and Regina’s online courses:
Demystifying Data: A Modern Approach to Statistical Understanding
Clinical Trials: Design, Strategy, and Analysis
Medical Statistics Certificate Program
Epidemiology and Clinical Research Graduate Certificate Program
Program that Kristin teaches in::
Epidemiology and Clinical Research Graduate Certificate Program
Kristin’s manuscript writing course:
https://instats.org/seminar/from-data-to-discussion-writing-a-scient
Find us on:
Kristin - LinkedIn & Twitter/X
Regina - LinkedIn & ReginaNuzzo.com
- (00:00) - Cold Open: Two Frauds in a Paper About Honesty
- (02:41) - The Claim: Spreadsheets Are the Villain
- (05:43) - The 2012 Honesty Pledge Study
- (09:17) - Signing at the Top Wins
- (13:31) - Failed Replications: Five Conceptual Tries, Then a Direct One
- (17:39) - How Data Colada Got the Data and Found Car Insurance Fraud
- (24:52) - Out-of-Order...
Kapitel
Kapitel werden geladen …
Statistical Sleuthing: Can you spot a problem?
Can you spot problems in a scientific study just by reading the published paper? We put our statistical sleuthing skills to the test with a famous study on procrastination and deadlines that was retracted more …
Shownotes
Can you spot problems in a scientific study just by reading the published paper? We put our statistical sleuthing skills to the test with a famous study on procrastination and deadlines that was retracted more than 20 years after it was published. But were there clues hidden in plain sight in the published results—and would we have noticed anything suspicious at the time? We put on our data-detective hats and find a few statistical clues of our own before turning to Data Colada’s investigation of the original data, where the story gets even stranger. Along the way, we hunt for missing participants, question suspiciously enormous effects, meet some unlikely data twins, and discover why sometimes the best statistical tool is simply asking: Do these data behave like real people?
Statistical topics
- Statistical sleuthing
- Effect sizes
- Degrees of freedom
- Replication
- Missing data
Methodologic Morals
- “A data detective's toolbox is surprisingly simple. Check the degrees of freedom, look for twins, question giant effects, and ask whether the data actually make human sense.”
- “You don't have to be a statistician to catch a statistical criminal.”
References
- Data Colada blog posts:
- Data and code: https://researchbox.org/7135
- Retraction Watch article: https://retractionwatch.com/2026/09/03/procrastination-study-duke-dan-ariely-psychological-science-data-colada-tampering-retraction/
- Original 2002 paper: https://journals.sagepub.com/doi/10.1111/1467-9280.00441
- Retraction notice: https://journals.sagepub.com/doi/10.1177/09567976261488042
- Hyndman K, Bisin A. Replication of "Procrastination, Deadlines, and Performance: Self-Control by Precommitment". Psychol Sci. 2026;37(8):557-571. doi:10.1177/09567976261460772
Kristin and Regina’s online courses:
Demystifying Data: A Modern Approach to Statistical Understanding
Clinical Trials: Design, Strategy, and Analysis
Medical Statistics Certificate Program
Epidemiology and Clinical Research Graduate Certificate Program
Program that Kristin teaches in::
Epidemiology and Clinical Research Graduate Certificate Program
Kristin’s manuscript writing course:
https://instats.org/seminar/from-data-to-discussion-writing-a-scient
Find us on:
Kristin - LinkedIn & Twitter/X
Regina - LinkedIn & ReginaNuzzo.com
- (00:00) - Introduction
- (02:15) - The Procrastination Paper and Study Two
- (08:51) - Suspiciously Tidy Results
- (16:18) - Study One: Real Classroom Stakes
- (20:06) - Missing Data and First Red Flags
- (28:17) - How Data Colada Got the Data
- (33:43) - When the Replication Failed
- (35:36) - Red Flags: Data Twins and Sanity Checks
- (47:21) - Study One Gets the Spreadsheet Treatment
- (51:45) - The Grading Scheme Unravels
- (57:51) - Manuscript Versions Reveal the Truth
- (01:08:11) - Rating the Claim and Methodological Morals
Kapitel
Kapitel werden geladen …
Ctrl-Z Awards: Will retracting your paper ruin your career?
What happens when you discover that your published paper is wrong? Does retracting it ruin your reputation—or could owning the mistake actually make things better? In this episode, we meet the inaugural winners …
Shownotes
What happens when you discover that your published paper is wrong? Does retracting it ruin your reputation—or could owning the mistake actually make things better? In this episode, we meet the inaugural winners of the Ctrl-Z Awards, which recognize scientists for correcting or retracting their own work. Behavioral ecologist Kate Laskowski takes us inside a scientific detective story involving suspicious spider data, three retracted papers, and the fear that going public might tank her career. Then computational social scientist Alyssa Smith tells us how one small error in a figure led her to retract the paper at the center of her PhD and job talk. Along the way, we talk about data forensics, the stigma around retractions, and why self-correcting scientists are the heart of science.
Statistical topics
- Data management and documentation
- Open science
- Reproducible workflows
- Research integrity
- Retractions
- Scientific misconduct
- Statistical forensics
Methodologic Morals
- “Science is self-correcting, which means scientists need to be self-correcting too.”
- “Good documentation is boring—right up until it saves you.”
- “If you want scientists to correct their mistakes, reward them for correcting their mistakes.”
- “I'd rather be considered an honest idiot than for people to wonder if I'm untrustworthy.”
References
- Retraction Watch: https://retractionwatch.com/
- Ctrl-Z Awards: https://retractionwatch.com/ctrl-z-award/
- Announcement of inaugural Ctrl-Z winners: https://centerforscientificintegrity.org/2026/08/04/2026-winners-ctrl-z-award-courage-retracting-own-work/
- Kate Laskowski’s blog post: https://laskowskilab.faculty.ucdavis.edu/2020/01/29/retractions/
- Kate Laskowski’s Data Detective Course syllabus
- Alyssa Smith’s site: https://asmithh.github.io/
- Laskowski KL, Montiglio PO, Pruitt JN. Individual and Group Performance Suffers from Social Niche Disruption [retracted in: Am Nat. 2020 Feb;195(2):393. doi: 10.1086/708066.]. Am Nat. 2016;187(6):776-785. doi:10.1086/686220
- Laskowski KL, Pruitt JN. Evidence of social niche construction: persistent and repeated social interactions generate stronger personalities in a social spider [retracted in: Proc Biol Sci. 2020 Jan 29;287(1919):20200077. doi: 10.1098/rspb.2020.0077.]. Proc Biol Sci. 2014;281(1783):20133166. Published 2014 Mar 26. doi:10.1098/rspb.2013.3166
- Modlmeier AP, Laskowski KL, DeMarco AE, et al. Persistent social interactions beget more pronounced personalities in a desert-dwelling social spider [retracted in: Biol Lett. 2020 Feb;16(2):20200062. doi: 10.1098/rsbl.2020.0062.]. Biol Lett. 2014;10(8):20140419. doi:10.1098/rsbl.2014.0419
- Smith AH, Green J, F Welles B, Lazer D. Emergent structures of attention on social media are driven by amplification and triad transitivity [retracted in: PNAS Nexus. 2026 May 07;5(5):pgag137. doi: 10.1093/pnasnexus/pgag137.]. PNAS Nexus. 2025;4(4):pgaf106. Published 2025 Apr 1. doi:10.1093/pnasnexus/pgaf106
Kristin and Regina’s online courses:
Demystifying Data: A Modern Approach to Statistical Understanding
Clinical Trials: Design, Strategy, and Analysis
Medical Statistics Certificate Program
Epidemiology and Clinical Research Graduate Certificate Program
Programs that we teach in:
Epidemiology and Clinical Research Graduate Certificate Program
Find us on:
Kristin - LinkedIn & Twitter/X
Regina - LinkedIn & ReginaNuzzo.com
Statistical Power: When is it useful?
Statistical power is one of the most important ideas in statistics—and one of the most misunderstood. In this episode, we explain what statistical power really means, why researchers need to think about it …
Shownotes
Statistical power is one of the most important ideas in statistics—and one of the most misunderstood. In this episode, we explain what statistical power really means, why researchers need to think about it before they collect their data, and what goes wrong when they calculate “post hoc power” after the study is over. Along the way, we hunt for UFOs, test Kristin’s psychic abilities, explore the ethics of underpowered studies, and discover why post hoc power is really just a p-value wearing a fake mustache. Then statistician and powerlifter Andrew Althouse joins us for the inside story of a years-long scientific feud over post hoc power, where we discuss what happens when being wrong about statistics turns into doubling down, angry letters to the editor, and debates about how science corrects itself.
Statistical topics
- Effect size
- False negatives
- P-values
- Post hoc power
- Power calculations
- Research ethics
- Sample size
- Scientific controversy
- Sensitivity analysis
- Statistical power
- Statistical significance
- Study design
- Type II error
Methodologic Morals
- “Big questions need studies powerful enough to answer them.”
- “Post hoc power is circular reasoning masquerading as new information.”
- “Know when to cut your losses. A mistake gets more expensive every time you double down.”
References
- Althouse AD, Chow ZR. Comment on “Post-hoc power: If you must, at least try to understand.” Ann Surg. 2019;270:e78-e79.
- Althouse AD. Post hoc power: Not empowering, just misleading. J Surg Res. 2021;259:A3-A6.
- Altman DG. Statistics and ethics in medical research: III. How large a sample? BMJ. 1980;281:1336-1338.
- Bababekov YJ, Hung YC, Hsu YT, et al. Is the power threshold of 0.8 applicable to surgical science?—empowering the underpowered study. J Surg Res. 2019;241:235-239.
- Bababekov YJ, Stapleton SM, Mueller JL, et al. A proposal to mitigate the consequences of type 2 error in surgical science. Ann Surg. 2018;267:621-622.
- Chang DC, Stapleton SM. Response: The proliferation and misinterpretation of “as safe as” statements in surgical science: A call for professional discourse to search for a solution. J Surg Res. 2021;259:A12-A15.
- Freiman JA, Chalmers TC, Smith H Jr, et al. The importance of beta, the type II error and sample size in the design and interpretation of the randomized control trial: Survey of 71 negative trials. N Engl J Med. 1978;299:690-694.
- Hoenig JM, Heisey DM. The abuse of power: The pervasive fallacy of power calculations for data analysis. Am Stat. 2001;55:19-24.
- Nuzzo RL. Statistical Power. PM&R. 2016;8(9):907-912. doi:10.1016/j.pmrj.2016.08.004
- Nuzzo RL. Post hoc Power. PM&R. 2021;13(4):422-424. doi:10.1002/pmrj.12476
Kristin and Regina’s online courses:
Demystifying Data: A Modern Approach to Statistical Understanding
Clinical Trials: Design, Strategy, and Analysis
Medical Statistics Certificate Program
Epidemiology and Clinical Research Graduate Certificate Program
Programs that we teach in:
Epidemiology and Clinical Research Graduate Certificate Program
Find us on:
Kristin - LinkedIn & Twitter/X
Regina - LinkedIn & ReginaNuzzo.com
- (00:00) - Intro
- (00:43) - What is Statistical Power?
- (03:22) - The UFO Telescope Analogy
- (06:19) - The Psychic Ability Experiment
- (10:36) - Planning Power Before You Study
- (12:46) - History and the Landmark 1978 Paper
- (17:42) - Enter Post-Hoc Power: The Problem
- (19:26) - Three Problems with Post-Hoc Power
- (24:33) - The Surgery Journal Disaster
- (29:59) - Guest Interview: Andrew Althouse
- (32:30) - The 2018 Paper and the Controversy Begins
- (39:54) - Doubling Down and the Cost of Being Right
- (53:03) - Rating the Claim
Kapitel
Kapitel werden geladen …
Cannabis and Brain Aging: Can weed keep your brain young?
Can weed protect your brain as you get older? We unpack a study linking lifetime cannabis use to larger brain volumes and better cognitive test scores in older adults—and investigate how those findings became …
Shownotes
Can weed protect your brain as you get older? We unpack a study linking lifetime cannabis use to larger brain volumes and better cognitive test scores in older adults—and investigate how those findings became headlines urging readers to “pass the pot to grandma.” Along the way, we explore collapsing categorical variables, confounding, clinically versus statistically significant results, multiple testing, and how a press release subtly changed a story. We also meet permanently stoned mice, learn about Carl Sagan’s surprising connection to cannabis research, and discover how a single missing word can change the entire meaning of a study.
Statistical topics
- Categorical variables
- Confounding
- Effect size
- False discovery rate
- Healthy volunteer bias
- Measurement error
- Multiple testing
- Observational studies
- Recoding variables
- Science communication
- Statistical vs practical significance
Methodologic Morals
- “Always follow the chain from data to result to conclusions and pay attention to the words along the way.”
- “Huge samples can give tiny differences delusions of grandeur.”
References
- Guha A, Fu Z, Calhoun V, Hutchison KE. Lifetime Cannabis Use Is Associated With Brain Volume and Cognitive Function in Middle-Aged and Older Adults. J Stud Alcohol Drugs. 2026;87(4):739-751. doi:10.15288/jsad.25-00346
- Press Release: https://news.cuanschutz.edu/news-stories/study-finds-cannabis-usage-in-middle-aged-and-older-adults-associated-with-larger-brain-volume-better-cognitive-function
- Bilkei-Gorzo A, Albayram O, Draffehn A, et al. A chronic low dose of Δ9-tetrahydrocannabinol (THC) restores cognitive function in old mice. Nat Med. 2017;23(6):782-787. doi:10.1038/nm.4311
- NY Post: https://nypost.com/2026/02/07/health/cannabis-may-benefit-aging-brains-study-finds/
- David Grinspoon: https://www.nytimes.com/2020/07/02/science/lester-grinspoon-dead.html?unlocked_article_code=1.7lA.CyS1.neeqGGNgx8g1&smid=url-share
Kristin and Regina’s online courses:
Demystifying Data: A Modern Approach to Statistical Understanding
Clinical Trials: Design, Strategy, and Analysis
Medical Statistics Certificate Program
Epidemiology and Clinical Research Graduate Certificate Program
Programs that we teach in:
Epidemiology and Clinical Research Graduate Certificate Program
Find us on:
Kristin - LinkedIn & Twitter/X
Regina - LinkedIn & ReginaNuzzo.com
- (00:00) - Opening: The Brain on Drugs
- (00:37) - The Cannabis Headline
- (03:38) - The Claim: Brain Benefits for Older Adults
- (06:53) - The Mouse Study: Pothead Mice vs. Control Mice
- (11:25) - From Mice to Humans: UK Biobank
- (15:01) - Measurement Problems and Category Collapse
- (21:39) - Results: Statistically Significant, Clinically Meaningless
- (27:13) - Individual Brain Regions and Cherry-Picking
- (32:32) - Cognitive Tests: What Actually Matters
- (40:09) - How It Gets Lost in Translation
- (45:32) - Rating the Claim
Kapitel
Kapitel werden geladen …
Double Duty: Can the meningitis vaccine also protect against gonorrhea?
Can the meningitis vaccine do double duty against gonorrhea? We examine a fascinating randomized trial inspired by years of encouraging observational studies—and what happened when those earlier results were …
Shownotes
Can the meningitis vaccine do double duty against gonorrhea? We examine a fascinating randomized trial inspired by years of encouraging observational studies—and what happened when those earlier results were finally put to the test. Along the way, we discuss ecological studies, case-control studies, why statisticians distrust phrases like “trend toward significance,” how intention-to-treat differs from per-protocol analysis, and why no amount of statistical adjustment can fully substitute for randomization. Along the way we celebrate outrageously named clinical trials, ask whether “flirting with statistical significance” belongs in a romance novel or a medical journal, and discover that “practice safe statistics” may be the best advice of the episode.
Statistical topics
- Case-control studies
- Confounding
- Ecological studies
- Generalizability
- Intention-to-treat
- Meta-analysis
- Null results
- Observational studies
- P-values
- Per-protocol analysis
- Randomized clinical trials
- Survival analysis
Methodologic Morals
- “Don't mistake a p-value of 0.06 for a p-value of 0.04 with bad luck.”
- “Practice safe statistics; randomize whenever possible.”
References
Seib KL, Donovan B, Jin F, et al. Meningococcal B Vaccine to Prevent Neisseria gonorrhoeae Infection. N Engl J Med. 2026; 395: 349-61.
Petousis-Harris H, Paynter J, Morgan J, et al. Effectiveness of a group B outer membrane vesicle meningococcal vaccine against gonorrhoea in New Zealand: a retrospective case-control study. The Lancet. 2017; 390: 1603–10.
Wang, B., Mohammed, H., Andraweera, P., McMillan, M., Marshall, H., 2024. Vaccine effectiveness and impact of meningococcal vaccines against gonococcal infections: A systematic review and meta-analysis. Journal of Infection. 2024; 89: 106225.
Molina JM, Bercot B, Assoumou, L, et al. Doxycycline prophylaxis and meningococcal group B vaccine to prevent bacterial sexually transmitted infections in France (ANRS 174 DOXYVAC): a multicentre, open-label, randomised trial with a 2 × 2 factorial design. The Lancet Infectious Diseases. 2024; 24: 1093–1104.
Thng C, Eskandari S, Jin F, et al. Efficacy of the meningococcal vaccine against Neisseria gonorrhoeae: a randomised clinical trial (MenGO). npj Vaccines. 2026.
Still Not Significant | Probable Error: List of funny disguises scientists use to describe p>.05Kristin and Regina’s online courses:
Demystifying Data: A Modern Approach to Statistical Understanding
Clinical Trials: Design, Strategy, and Analysis
Medical Statistics Certificate Program
Epidemiology and Clinical Research Graduate Certificate Program
Programs that we teach in:
Epidemiology and Clinical Research Graduate Certificate Program
Find us on:
Kristin - LinkedIn & Twitter/X
Regina - LinkedIn & ReginaNuzzo.com
- (00:15) - - Introduction
- (00:51) - - The Claim: Can a Meningitis Vaccine Prevent Gonorrhea?
- (04:10) - - The Biology Behind the Idea
- (10:37) - - How Observational Studies Made the Link
- (14:44) - - Earlier Clinical Trials: MENGO and DOXYVAC
- (17:30) - - Flirting with Significance: A Statistical Rant
- (22:40) - - GoGoVax: Study Design and Analysis
- (33:31) - - The Shocking Null Result
- (35:38) - - Why These Results Don't Generalize
- (37:42) - - Rating the Claim
Kapitel
Kapitel werden geladen …
Hot Tubbing: Can soaking in hot water help you run faster?
Can you improve your endurance by sitting in a hot tub instead of doing another hard workout? We dig into a study that put elite runners through five weeks of what we affectionately call “endurance hot tubbing” …
Shownotes
Can you improve your endurance by sitting in a hot tub instead of doing another hard workout? We dig into a study that put elite runners through five weeks of what we affectionately call “endurance hot tubbing” to see whether heat alone could trigger the same cardiovascular adaptations as altitude training. Along the way, we explore crossover study designs, multiple testing, why significant physiological changes don’t necessarily translate into better athletic performance, and how an exploratory regression analysis ended up answering the wrong scientific question. We also discover that serious hot tub research involves surprisingly little relaxation, debate whether bubbles should count as an experimental condition, and learn why distance runners pay good money to sleep in tents with some of the oxygen sucked out.
Statistical topics
- best subsets regression
- causal inference
- crossover studies
- exploratory analyses
- mediation / mechanisms
- model specification
- multiple testing
- peer review
- sample size
- (00:15) - Introduction
- (00:44) - Can Hot Tubs Replace Exercise?
- (01:27) - Crazy Runners and Crazy Training
- (05:44) - The Amazon Delivery System Metaphor
- (12:12) - From Heat Training to Hot Tubs
- (13:33) - The Study: 10 Runners in Hot Water
- (22:31) - Results and the VO2 Max Question
- (29:26) - Statistical Sleuthing: Best Subsets Regression
- (36:50) - The Peer Review Problem
- (41:39) - Rating the Claim
Methodologic Morals
- “A model can be very good at answering the wrong question.”
- “The label ‘peer reviewed’ does not guarantee ‘carefully reviewed’.”
References
- Jenkins EJ, Killick JA, Zerilli O, et al. Long-term passive heat acclimation enhances maximal oxygen consumption via haematological and cardiac adaptation in endurance runners. J Physiol. Published online November 20, 2025. doi:10.1113/JP289874
- Stembridge M, Jenkins E. Marathon training: Why hot baths might help you run faster. The Conversation. Published March 16, 2026. doi:10.64628/AB.6vknm9esh
Kristin and Regina’s online courses:
Demystifying Data: A Modern Approach to Statistical Understanding
Clinical Trials: Design, Strategy, and Analysis
Medical Statistics Certificate Program
Epidemiology and Clinical Research Graduate Certificate Program
Programs that we teach in:
Epidemiology and Clinical Research Graduate Certificate Program
Find us on:
Kristin - LinkedIn & Twitter/X
Regina - LinkedIn & ReginaNuzzo.com
- (00:15) - Introduction
- (00:44) - Can Hot Tubs Replace Exercise?
- (01:27) - Crazy Runners and Crazy Training
- (05:44) - The Amazon Delivery System Metaphor
- (12:12) - From Heat Training to Hot Tubs
- (13:33) - The Study: 10 Runners in Hot Water
- (22:31) - Results and the VO2 Max Question
- (29:26) - Statistical Sleuthing: Best Subsets Regression
- (36:50) - The Peer Review Problem
- (41:39) - Rating the Claim
Kapitel
Kapitel werden geladen …
Exercise Snacks: Can just four minutes of exercise a day improve blood sugar?
Could four one-minute bursts of exercise really improve blood sugar? We try “exercise snacks” ourselves before taking a close look at the clinical trial that inspired headlines. We explain why the study’s main …
Shownotes
Could four one-minute bursts of exercise really improve blood sugar? We try “exercise snacks” ourselves before taking a close look at the clinical trial that inspired headlines. We explain why the study’s main result wasn’t statistically significant, how 34 secondary outcomes complicated the story, and what pre-registration can reveal about a study after it’s published. Along the way, we compare notes on our own exercise-snacking adventures, debate continuous glucose monitors, and ask how much evidence a single study should generate before it becomes health news.
Statistical topics
- Crossover design
- Multiple testing
- Pre-registration
- Primary vs secondary outcomes
- Randomized controlled trial
- Research transparency
Methodologic Morals
- “It's good relationship advice to be transparent. It's also good research advice.”
- “If the primary outcome is not significant, say it up top.”
References
- Babir FJ, Marcotte-Chénard A, Sandilands RE, et al. Exercise snacks performed in real-world settings reduce postprandial hyperglycaemia and glycaemic variability in individuals living with type 2 diabetes: a randomised crossover study. Diabetologia. 2026;69(8):2200-2211. doi:10.1007/s00125-026-06741-2
- https://www.washingtonpost.com/wellness/2026/05/28/4-minutes-exercise-day-could-help-control-blood-sugar/
- clinicaltrials.gov pre-registration with changes: https://clinicaltrials.gov/study/NCT06382246?term=NCT06382246&rank=1&tab=history&a=1&b=2#version-content-panel
Kristin and Regina’s online courses:
Demystifying Data: A Modern Approach to Statistical Understanding
Clinical Trials: Design, Strategy, and Analysis
Medical Statistics Certificate Program
Epidemiology and Clinical Research Graduate Certificate Program
Programs that we teach in:
Epidemiology and Clinical Research Graduate Certificate Program
Find us on:
Kristin - LinkedIn & Twitter/X
Regina - LinkedIn & ReginaNuzzo.com
- (00:00) - Intro
- (04:06) - The claim: four minutes a day
- (07:05) - Our own N of 1 experiments
- (13:24) - The study
- (21:54) - Primary outcome: complete miss
- (25:16) - Secondary outcomes to the rescue?
- (35:02) - Statistical sleuthing and transparency
- (44:23) - Rating the claim
Kapitel
Kapitel werden geladen …
Cancer Blood Tests Part 2: The clinical trial
How do you decide whether a clinical trial “worked”? In Part 2 of our Galleri series, we examine the landmark randomized trial of a blood test designed to detect more than 50 cancers. We explore why different …
Shownotes
How do you decide whether a clinical trial “worked”? In Part 2 of our Galleri series, we examine the landmark randomized trial of a blood test designed to detect more than 50 cancers. We explore why different outcome measures led to dramatically different headlines, discuss primary versus secondary outcomes, pre-registration, hierarchical testing, and post hoc analyses, and explain why mortality remains the outcome everyone is waiting for. Along the way, we uncover a statistical mystery involving dozens of missing cancers and discover how a little arithmetic can sometimes reveal more than a press release.
Statistical topics
- cancer screening
- exploratory analyses
- hierarchical testing
- missing data
- multiple testing
- outcome measures
- post hoc analyses
- pre-registration
- primary and secondary outcomes
- randomized clinical trials
- screening tests
Methodologic Morals- “When the simple numbers don't add up, pay attention. The arithmetic may be trying to tell you something.”
- “The first question should not be, did it work? It should be, what counts as success?”
References- Giridhar KV, et al. Safety and performance results from PATHFINDER 2, a registrational study of a multi-cancer early detection test in an intended-use population. Presented at the 2026 American Society of Clinical Oncology (ASCO) Annual Meeting. May 2026.
- Hubbell E, Clarke CA, Aravanis AM, Berg CD. Modeled Reductions in Late-stage Cancer with a Multi-Cancer Early Detection Test. Cancer Epidemiol Biomarkers Prev. 2021;30(3):460-468. doi:10.1158/1055-9965.EPI-20-1134
- Neal RD, Johnson P, Clarke CA, et al. Cell-Free DNA-Based Multi-Cancer Early Detection Test in an Asymptomatic Screening Population (NHS-Galleri): Design of a Pragmatic, Prospective Randomised Controlled Trial. Cancers (Basel). 2022;14(19):4818. Published 2022 Oct 1. doi:10.3390/cancers14194818
- ASCO slides: https://grail.com/wp-content/uploads/2026/05/Swanton_ASCO-2026_NHS-Galleri_FINAL-Slides-05.26.2026.pdf
- UK registry protocol: https://www.isrctn.com/ISRCTN91431511
- Clinicaltrials.gov protocol: https://clinicaltrials.gov/study/NCT05611632
Common biases in cancer screening studiesCancer screening studies are subject to several well-known biases that can make a screening test appear more effective than it actually is. Three of the most important are:
Lead-time bias: Screening advances the time of diagnosis, making survival from diagnosis appear longer even if the patient's lifespan is unchanged. For example, if a screening test detects a Stage II cancer at age 60 that otherwise would have been diagnosed because of symptoms at age 62, but the patient dies at age 68 regardless, survival from diagnosis appears to increase from 6 years to 8 years even though the patient did not live any longer.
Length bias: Screening preferentially detects slower-growing, less aggressive cancers because they remain detectable for longer than fast-growing cancers. For example, a slow-growing cancer that remains in Stage I for 5 years is much more likely to be found by screening than an aggressive cancer that progresses to symptoms within months. This can make screened patients appear to have better survival simply because screening preferentially found the less aggressive cancers.
Overdiagnosis: Screening detects cancers that would never have caused symptoms or death during a person's lifetime, leading to unnecessary diagnosis and treatment. For example, a screening test may detect a very slow-growing prostate or thyroid cancer in an older adult that would never have become clinically important if it had remained undiscovered.
Kristin and Regina’s online courses:
Demystifying Data: A Modern Approach to Statistical Understanding
Clinical Trials: Design, Strategy, and Analysis
Medical Statistics Certificate Program
Epidemiology and Clinical Research Graduate Certificate Program
Programs that we teach in:
Epidemiology and Clinical Research Graduate Certificate Program
Find us on:
Kristin - LinkedIn & Twitter/X
Regina - LinkedIn & ReginaNuzzo.com
- (00:00) - Intro
- (03:39) - The Claim: Not Ready for Primetime
- (03:58) - Trial Design: 142,000 Participants
- (07:50) - The Primary Outcome Problem
- (20:29) - The Primary Endpoint: Complete Miss
- (22:14) - Three Arguments for the Defense
- (28:29) - - Statistical Sleuthing: Missing Cancers
- (41:14) - - The Stage Shift Argument
- (50:30) - - Rating the Claim
Kapitel
Kapitel werden geladen …
Cancer Blood Tests: Are they ready for primetime? Part 1
Can a single tube of blood really detect dozens of cancers before symptoms appear? We dive into the science behind Galleri, a blood test that claims to detect more than 50 types of cancer from a simple blood …
Shownotes
Can a single tube of blood really detect dozens of cancers before symptoms appear? We dive into the science behind Galleri, a blood test that claims to detect more than 50 types of cancer from a simple blood draw. Recent headlines about the test ranged from “breakthrough” to “bust” after the release of results from a massive randomized clinical trial. In this Part 1 episode, we explore cell-free DNA, DNA methylation, machine learning, sensitivity, specificity, and positive predictive value. Along the way, we revisit the prenatal screening revolution, ask why detecting cancer earlier doesn’t always help patients, and learn how escaped DNA convicts end up swimming in a giant molecular pool party. And for the first time ever, Normal Curves ends on a cliffhanger: we’ll save the controversial results of that landmark trial for Part 2.
Statistical topics
- cancer screening
- case-control studies
- counterfactuals
- machine learning
- negative predictive value
- overdiagnosis
- positive predictive value
- randomized clinical trials
- screening tests
- sensitivity and specificity
- validation
References
- Bianchi DW, Chudova D, Sehnert AJ, et al. Noninvasive prenatal testing and incidental detection of occult maternal malignancies. JAMA. 2015; 314:162-9.
- Liu MC, Oxnard GR, Klein EA, et al. Sensitive and specific multi-cancer detection and localization using methylation signatures in cell-free DNA. Ann Oncol. 2020. 31:745-59.
- Schrag D, Beer T, McDonnell C et al. Blood-based tests for multicancer early detection (PATHFINDER): a prospective cohort study. The Lancet. 402: 1251-60.
- Giridhar KV, et al. Safety and performance results from PATHFINDER 2, a registrational study of a multi-cancer early detection test in an intended-use population. Presented at the 2026 American Society of Clinical Oncology (ASCO) Annual Meeting. May 2026.
Statistic discussed in the episodePATHFINDER 2 investigators reported that adding Galleri to routine screening increased the number of screen-detected cancers by 6.5-fold. This figure compares 31 cancers detected through USPSTF-recommended screening (for breast, cervical, lung, and colon) with 204 cancers detected when Galleri was added, counting the same 31 conventional-screening cancers in both totals. Thus, describing the increase as 6.5-fold is misleading, since the combination of Galleri plus conventional screening is, by definition, guaranteed to detect at least as many cancers as conventional screening alone. Moreover, everyone in the study received Galleri, whereas conventional screening depended on which tests participants happened to be due for and completed during the study period. The comparison therefore does not involve two equally applied screening strategies.
Kristin and Regina’s online courses:Demystifying Data: A Modern Approach to Statistical Understanding
Clinical Trials: Design, Strategy, and Analysis
Medical Statistics Certificate Program
Epidemiology and Clinical Research Graduate Certificate Program
Programs that we teach in:
Epidemiology and Clinical Research Graduate Certificate Program
Find us on:
Kristin - LinkedIn & Twitter/X
Regina - LinkedIn & ReginaNuzzo.com
- (00:00) - - Introduction
- (00:44) - - The Holy Grail of Cancer Testing
- (04:31) - - Headlines: Same Data, Opposite Stories
- (07:38) - - How Cell-Free DNA Works
- (13:54) - - DNA Methylation: GRAIL's Fingerprint
- (15:19) - - The Origin Story
- (22:18) - - The Pathfinder Studies
- (35:01) - - The Paradox: Why Earlier Detection Doesn't Always Help
- (40:32) - - The Cliffhanger
Kapitel
Kapitel werden geladen …
Odds Ratios: Do most people get them wrong?
Odds ratios show up everywhere in medical research—but do readers, journalists, and even researchers always know what they mean? In this episode, we tackle one of the most common statistical misunderstandings …
Shownotes
Odds ratios show up everywhere in medical research—but do readers, journalists, and even researchers always know what they mean? In this episode, we tackle one of the most common statistical misunderstandings in science: treating odds ratios like risk ratios. Along the way, we explore puppy photos, fish photos, first-date hookups, sugary drinks, cardiac care, and a listener challenge that started with an informal study of five medical residents and a box of chocolate truffles. We explain why logistic regression produces odds ratios, when odds ratios can wildly exaggerate effects, and why some famous headlines turned out to be much less dramatic than they sounded.
Statistical topics- binary outcomes
- case-control studies
- logistic regression
- odds ratios
- risk ratios
- odds vs risk
Methodological morals- “Just because logistic regression gives you an odds ratio does not mean you have to report it.”
- “A lot of bad science communication starts long before the journalist even enters the story.”
References
- Bleich SN, Herring BJ, Flagg DD, et al. Reduction in purchases of sugar-sweetened beverages among low-income Black adolescents after exposure to caloric information. Am J Public Health. 2012;102:329–35.
- Sainani KL. How Statistics Can Mislead. Am J Public Health. 2012. 2012;102:e3–e4.
- Bleich SN, Herring BJ, Flagg DD, et al. Bleich et al. respond. Am J Public Health. 2012;102:e4.
- Press video: https://www.youtube.com/watch?v=IFyrqbf1XWs
- Sainani KL, Schmajuk G, Liu V. A Caution on Interpreting Odds Ratios. Sleep. 2009;32:976.
- Schulman KA, Berlin JA, Harless W, et al. The Effect of Race and Sex on Physicians' Recommendations for Cardiac Catheterization. NEJM. 1999;340:618–26.
- Schwartz LM, Woloshin S, Welch HG. Misunderstandings about the Effects of Race and Sex on Physicians' Referrals for Cardiac Catheterization. NEJM. 1999;341:279–83.
- Associated Press. Study Finds Bias in Doctors' Care of Women and Blacks. The New York Times. February 25, 1999.
- Knol MJ, Duijnhoven RG, Grobbee DE, et al. Potential Misinterpretation of Treatment Effects Due to Use of Odds Ratios and Logistic Regression in Randomized Controlled Trials. PLoS ONE. 2011;6:e21248.
More information on logistic regression and odds ratios:
- Sainani KL. Logistic Regression. PM&R. 2014;6:1157–62.
- Sainani KL. Understanding Odds Ratios. PM&R. 2011;3:263–67.
- Nuzzo RL. Communicating measures of relative risk in plain English. PM&R. 2022;14:283-287.
When outcomes are common, odds ratios can exaggerate effect sizes. Alternatives include:
- Presenting raw percentages (absolute risks)
- Presenting adjusted percentages from logistic regression (these may be calculated by plugging in means for the covariates)
- Converting odds ratios to risk ratios
- Reporting risk ratios directly when appropriate
Converting Odds Ratios to Risk Ratios:
- Zhang J, Yu KF. What's the Relative Risk? A Method of Correcting the Odds Ratio in Cohort Studies of Common Outcomes. JAMA. 1998;280:1690–91.
- ClinCalc. Odds Ratio to Relative Risk Calculator.
- https://clincalc.com/stats/convertor.aspx
- RR = OR / [(1 − P0) + (P0 × OR)]
Example:
OR=0.51, baseline risk=93.3%
RR = 0.51 / [(1 − 0.933) + (0.933 × 0.51)]
= 0.51 / (0.067 + 0.476)
= 0.51 / 0.543
= 0.94Thus, an odds ratio of 0.51 corresponds to a risk ratio of 0.94 when the baseline risk is 93.3%.
The corresponding unadjusted risk ratio is 86%/93.3%=0.92
Correction: In the episode, we stated that the adjusted risk ratio was 0.92. In fact, it is 0.94, as shown above. 0.92 is the unadjusted risk ratio.
Kristin and Regina’s online courses:Demystifying Data: A Modern Approach to Statistical Understanding
Clinical Trials: Design, Strategy, and Analysis
Medical Statistics Certificate Program
Epidemiology and Clinical Research Graduate Certificate Program
Programs that we teach in:
Epidemiology and Clinical Research Graduate Certificate Program
Find us on:
Kristin - LinkedIn & Twitter/X
Regina - LinkedIn & ReginaNuzzo.com
- (00:00) - Introduction
- (02:54) - What Are Odds Ratios?
- (04:02) - Puppy Photos and First Dates
- (06:09) - Risk Ratio Explained
- (08:10) - Calculating Odds Ratios
- (11:09) - Fish Photos and Reversed Numbers
- (16:01) - Real-Life Example: Sugary Beverages
- (22:08) - How Logistic Regression Works
- (31:53) - The Video: Researchers Made the Mistake Themselves
- (36:30) - The Cardiac Catheterization Study
- (39:24) - The New York Times Printed a Correction
- (46:10) - Using OR and RR Interchangeably for Case Control
- (47:00) - Reye Syndrome and Aspirin
- (49:37) - Rating the Claim and Methodological Morals
Kapitel
Kapitel werden geladen …
Coffee and the Heart: Is caffeine a trigger for AFib?
Does coffee trigger atrial fibrillation — or have doctors been warning people away from caffeine without strong evidence? We dig into two recent randomized clinical trials testing whether caffeinated coffee …
Shownotes
Does coffee trigger atrial fibrillation — or have doctors been warning people away from caffeine without strong evidence? We dig into two recent randomized clinical trials testing whether caffeinated coffee causes dangerous heart rhythm problems. Along the way, we talk about AFib, survival analysis, intention-to-treat versus as-treated analyses, and one surprisingly elaborate effort to catch clinical trial cheaters with receipts and geolocation tracking. We also explore how a pope may have fueled a European coffee resurgence, why plants make caffeine, and how a game show competition explains hazard ratios.
Statistical topics- adherence and compliance
- as-treated analysis
- confidence intervals
- Cox proportional hazards regression
- hazard ratios
- intention-to-treat analysis
- micro-randomization
- multiple testing
- PICOT
- pre-registration
- primary vs secondary outcomes
- randomized clinical trials
- sensitivity analyses
- SMART framework
- survival analysis
Methodological morals- “Never trust conventional wisdom until you see the randomized controlled trial.”
- “Trust your participants, but design the study so that they can be honest about their dishonesty.”
References- Harrington D, D'Agostino RB Sr, Gatsonis C, et al. New Guidelines for Statistical Reporting in the Journal. N Engl J Med. 2019;381(3):285-286. doi:10.1056/NEJMe1906559
- Marcus GM, Rosenthal DG, Nah G, et al. Acute Effects of Coffee Consumption on Health among Ambulatory Adults. N Engl J Med. 2023;388(12):1092-1100. doi:10.1056/NEJMoa2204737
- Wong CX, Cheung CC, Montenegro G, et al. Caffeinated Coffee Consumption or Abstinence to Reduce Atrial Fibrillation: The DECAF Randomized Clinical Trial. JAMA. 2026;335(4):317-325. doi:10.1001/jama.2025.21056
- @MarcKatzMD’s short video The Pitt- atrial fibrillation cardioversion scene
Kristin and Regina’s online courses:Demystifying Data: A Modern Approach to Statistical Understanding
Clinical Trials: Design, Strategy, and Analysis
Medical Statistics Certificate Program
Epidemiology and Clinical Research Graduate Certificate Program
Programs that we teach in:
Epidemiology and Clinical Research Graduate Certificate Program
Find us on:
Kristin - LinkedIn & Twitter/X
Regina - LinkedIn & ReginaNuzzo.com
- (00:00) - - Introduction
- (02:15) - - What is AFib?
- (04:36) - - Frisky Goats and Satan's Bitter Invention
- (10:44) - - How Caffeine Works
- (14:43) - - The CRAVE Trial
- (15:53) - - PICOT: Evaluating the Study Design
- (23:24) - - CRAVE Results
- (30:07) - - Catching the Coffee Cheaters
- (37:01) - - The DECAF Trial
- (41:30) - - Time-to-Event Outcomes
- (43:21) - - Hazard Ratios: Balance Beams Over Shark Tanks
- (47:06) - - DECAF Results: Team Coffee Wins
- (50:38) - - Why Would Coffee Be Protective?
- (53:57) - - Rating the Claim
Kapitel
Kapitel werden geladen …
Sleep and Exercise: Does working out on too little sleep speed up aging?
Can exercise actually be bad for you if you don’t get enough sleep? A widely shared claim says yes—that working out while sleep deprived may speed up aging. In this episode, we put that claim under the …
Shownotes
Can exercise actually be bad for you if you don’t get enough sleep? A widely shared claim says yes—that working out while sleep deprived may speed up aging. In this episode, we put that claim under the microscope. We examine the study behind it, unpack how sleep and aging were measured, and explore key statistical ideas like interaction effects and flexible models that can “dance” to the data. With the help of a $400,000 handbag and a man with seven boats, we also break down what it really takes to show that one variable changes the effect of another. What we find: some clear study bloopers, inconsistent modeling results, and interpretations that are flat-out wrong.
Statistical topics- Measurement error
- Model specification
- Piecewise linear regression
- Regression models
- Residual confounding
- Splines
- Statistical interactions
- Survey design
Methodological morals- “Before you believe something shocking, ask what had to go wrong to make it true.”
- “If slight modeling changes flip the story, there wasn't much story to begin with.”
- “Unethical Life Pro Tip: If you do not want your analysis critiqued, then just make it impossible to understand.”
Kristin’s Biological Age Calculator
References- Original Viral Tweet: Ng D. "People who slept under 6 hours and exercised actually aged faster." X. March 9, 2026.
- Holmer B. Does exercise “age you faster” if you don’t sleep enough? Medium. March 16, 2026.
- You Y. Chen Y. Liu R., et al. Inverted U-shaped relationship between sleep duration and phenotypic age in US adults: a population-based study. Sci Rep. 2024;14:6247.
- Levine ME, Lu AT, Quach A, et al. An epigenetic biomarker of aging for lifespan and healthspan. Aging. 2018;10:573-591.
Kristin and Regina’s online courses:Demystifying Data: A Modern Approach to Statistical Understanding
Clinical Trials: Design, Strategy, and Analysis
Medical Statistics Certificate Program
Epidemiology and Clinical Research Graduate Certificate Program
Programs that we teach in:
Epidemiology and Clinical Research Graduate Certificate Program
Find us on:
Kristin - LinkedIn & Twitter/X
Regina - LinkedIn & ReginaNuzzo.com
- (00:00) - Introduction
- (04:05) - What is NHANES?
- (06:38) - The Sleep Duration Results
- (12:50) - The 2015 Sleep Mystery
- (17:10) - Measuring Biological Aging
- (21:35) - The Penalized Cox Regression
- (28:16) - Sleep and Aging Results
- (30:03) - Cubic Splines and Dancing
- (36:49) - Adding Exercise to the Mix
- (40:57) - Boats, Handbags, and Interaction Effects
- (48:20) - The Cubic Spline Exercise Analysis
- (51:21) - The Opposite Result
- (55:54) - Academic Writing Gone Wrong
- (58:27) - The Writing Makeover
- (01:01:12) - Rating the Claim with Gatorinis
Kapitel
Kapitel werden geladen …
Sex Recession: Are young people really having less sex?
Are young people really having less sex? Headlines about a “sex recession” suggest a dramatic decline—but what do the data actually show? In this episode, we trace that claim back to the research behind it—and …
Shownotes
Are young people really having less sex? Headlines about a “sex recession” suggest a dramatic decline—but what do the data actually show? In this episode, we trace that claim back to the research behind it—and find a story that’s far more nuanced than the headlines suggest. We examine large national surveys, including the General Social Survey and the National Survey of Sexual Health and Behavior, and uncover how small analytical choices can completely change the story. Along the way, we tackle ordinal versus quantitative data, why averages can mislead, how logistic regression reframes the question, and what happens when researchers try to time-travel with statistics. Plus: the surprising role of extreme values, why “eight fewer sexual encounters per year” may not mean what you think, and whether young men and women are really following the same trends.
Statistical topics- Average vs distribution
- Binary variables
- Effect size vs statistical significance
- Logistic regression
- Measurement / operationalization
- Ordinal variables
- Outliers / extreme values
- Self-reported datagoog
- Social desirability bias
- Variable coding / transformation
Methodological morals- “You shouldn't use data from people in their 80s to guess what they were doing in their 20s unless your data come with a time machine.”
- “When extreme values drive the average, the average stops describing most people.”
References- Julian K. Why are young people having so little sex? The Atlantic. December 2018. Accessed April 19, 2026. https://www.theatlantic.com/magazine/archive/2018/12/the-sex-recession/573949/
- Skwarecki B. Nearly half of Gen Z adults have never had sex: report. Newsweek. January 7, 2025. Accessed April 19, 2026. https://www.newsweek.com/nearly-half-of-gen-z-adults-have-never-had-sexreport-11052178
- Virginity survey. DatingAdvice.com. Accessed April 19, 2026. https://www.datingadvice.com/studies/virginity-survey
- Twenge JM, Sherman RA, Wells BE. Declines in sexual frequency among American adults, 1989-2014. Arch Sex Behav. 2017;46(8):2389-2401.
- Ueda P, Mercer CH, Ghaznavi C, Herbenick D. Trends in frequency of sexual activity and number of sexual partners among adults aged 18 to 44 years in the US, 2000-2018. JAMA Netw Open. 2020;3(6):e203833.
- Herbenick D, Rosenberg M, Golzarri-Arroyo L, et al. Changes in penile-vaginal intercourse frequency and sexual repertoire from 2009 to 2018: findings from the National Survey of Sexual Health and Behavior. Arch Sex Behav. 2022;51(3):1419-1433.
- Wellings K, Palmer MJ, Machiyama K, Slaymaker E. Changes in, and factors associated with, frequency of sex in Britain: evidence from three National Surveys of Sexual Attitudes and Lifestyles (Natsal). BMJ. 2019;365:l1525. Published 2019 May 7. doi:10.1136/bmj.l1525
- Burghardt J, Beutel ME, Hasenburg A, Schmutzer G, Brähler E. Declining Sexual Activity and Desire in Women: Findings from Representative German Surveys 2005 and 2016. Arch Sex Behav. 2020 Apr;49(3):919-925. doi: 10.1007/s10508-019-01525-9. Epub 2019 Dec 4. Erratum in: Arch Sex Behav. 2020 Apr;49(3):927. doi: 10.1007/s10508-019-01622-9. PMID: 31802290.
- Twenge JM. Possible Reasons US Adults Are Not Having Sex as Much as They Used To. JAMA Netw Open. 2020;3(6):e203889. Published 2020 Jun 1. doi:10.1001/jamanetworkopen.2020.3889
Kristin and Regina’s online courses:Demystifying Data: A Modern Approach to Statistical Understanding
Clinical Trials: Design, Strategy, and Analysis
Medical Statistics Certificate Program
Epidemiology and Clinical Research Graduate Certificate Program
Programs that we teach in:
Epidemiology and Clinical Research Graduate Certificate Program
Find us on:
Kristin - LinkedIn & Twitter/X
Regina - LinkedIn & ReginaNuzzo.com
- (00:00) - Introduction
- (04:04) - Fact-Checking the Headlines
- (07:37) - The Twenge Study and the GSS
- (16:02) - The Hill-Shaped Trend
- (19:23) - The Ordinal Variable Problem
- (24:59) - The Married vs. Never-Married Paradox
- (27:42) - Time-Traveling to the 1920s
- (31:38) - The Ueda Study: A Better Approach
- (35:25) - The Two Classrooms
- (42:42) - What Counts as Sex?
- (48:52) - Historical Sex Terms
- (52:35) - The Sexual Repertoire Results
- (55:53) - Why Is This Happening?
- (01:02:12) - Rating the Claim
Kapitel
Kapitel werden geladen …
Diagnostic Testing: Do the stats tell you what you need to know?
Diagnostic testing: what do those statistics actually tell you? Sensitivity, specificity, positive predictive value . . . you’ve probably seen these terms before. Maybe you memorized them for a test. But do you …
Shownotes
Diagnostic testing: what do those statistics actually tell you? Sensitivity, specificity, positive predictive value . . . you’ve probably seen these terms before. Maybe you memorized them for a test. But do you actually know what they mean? In this episode, we take a closer look at how diagnostic tests are evaluated—and how they’re often misinterpreted. From a genetic test for cellulite to a blood test for autism, we explore how “statistically significant” findings can turn into tests that don’t actually help anyone. Along the way we meet the freckle gene, the wanderlust gene, and infidelity gene.
Statistical topics- Base Rate
- Bayes Rule
- Case-Control Study
- Matching
- Conditional Probability
- Sensitivity
- Specificity
- Positive Predictive Value
- Prevalence
- Negative Predictive Value
- False Positives and Negatives
- True Positives and Negatives
Methodological morals- “A biomarker paper is not the same thing as a biomarker test.”
- “If your sample doesn't match the real world, then for all of your positive predictive value needs, call on Bayes' theorem.”
Detailed Show Notes with calculations
References- Emanuele E, Bertona M, Geroldi D. A multilocus candidate approach identifies ACE and HIF1A as susceptibility genes for cellulite. Journal of the European Academy of Dermatology and Venereology; 2010. 24: 930-35.
- https://genomelink.io/traits/cellulite
- https://www.genexdiagnostics.com/
- Ebstein RP, Novick O, Umansky R, et al. Dopamine D4 receptor (D4DR) exon III polymorphism associated with the human personality trait of Novelty Seeking. Nat Genet. 1996;12:78-80.
- Kluger AN, Siegfried Z, Ebstein RP. A meta-analysis of the association between DRD4 polymorphism and novelty seeking. Mol Psychiatry. 2002;7:712-7.
- He Y, Martin N, Zhu G, Liu Y. Candidate genes for novelty-seeking: a meta-analysis of association studies of DRD4 exon III and COMT Val158Met. Psychiatr Genet. 2018 Dec;28(6):97-109.
- Smith AM, King JJ, West PR, et al. Amino Acid Dysregulation Metabotypes: Potential Biomarkers for Diagnosis and Individualized Treatment for Subtypes of Autism Spectrum Disorder. Biol Psychiatry. 2019;85:345-54.
- Sainani K, Goodman S. Lack of Diagnostic Utility of “Amino Acid Dysregulation Metabotypes.”
Biol Psychiatry. 2018; 85: e41-e42.
Kristin and Regina’s online courses
- Demystifying Data: A Modern Approach to Statistical Understanding
- Clinical Trials: Design, Strategy, and Analysis
- Medical Statistics Certificate Program
- Writing in the Sciences
- Epidemiology and Clinical Research Graduate Certificate Program
- Programs that we teach in:
- Epidemiology and Clinical Research Graduate Certificate Program
Find us on:
Kristin - LinkedIn & Twitter/X
Regina - LinkedIn & ReginaNuzzo.com
- (00:00) - Introduction
- (02:24) - The Cellulite Test
- (06:41) - Understanding Sensitivity and Specificity
- (12:50) - Enter Positive Predictive Value
- (18:40) - Why Base Rates Matter
- (24:06) - More Ridiculous Tests
- (32:33) - The Wanderlust Gene Deep Dive
- (40:30) - The NeuroPoint Autism Test
- (51:37) - Trying to Set the Record Straight
- (01:00:42) - Personal Stories
- (01:03:57) - Wrap-up
Kapitel
Kapitel werden geladen …
Epidurals: Are labor epidurals really linked to autism?
Epidurals are widely used and widely trusted for pain relief during labor. So when a 2020 study reported that they might be linked to autism, it raised a troubling question: could a routine medical decision …
Shownotes
Epidurals are widely used and widely trusted for pain relief during labor. So when a 2020 study reported that they might be linked to autism, it raised a troubling question: could a routine medical decision have long-term consequences? We follow that claim from headline to evidence—and watch what happens when other scientists take a closer look. We dig into the original study, a wave of replication studies from around the world, and a meta-analysis that tries to make sense of it all. Along the way, we unpack hazard ratios, Cox regression, inverse probability weighting, and sibling analyses—and why even sophisticated statistical adjustment can’t eliminate confounding. Plus: why bigger datasets don’t solve everything, what happens when results shrink after adjustment, and how a controversial study turned into a case study in science working as it should. Bonus: our first guest journalist interview!
Statistical topics
- Confounding
- Cox regression
- Hazard ratios
- Inverse probability weighting (IPTW)
- Multivariable adjustment
- Observational studies
- Residual confounding
- Retrospective cohort studies
- Sibling analysis
- Statistical adjustment
- Statistical significance vs practical significance
- Survival analysis
Methodological morals
- “Every time you adjust the model and the effect gets smaller, that's the universe whispering, maybe don't build a causal story out of this.”
- “Consistency across studies is gold.”
- “There's more to the story than the statistics.”
References- Dattaro, Laura. A questionable study linked autism to epidurals. Then what? Spectrum. April 18, 2023.
- Dattaro, Laura. How to find baby sharks. Nautilus. September 9. 2024.
- Laura Dattaro’s home page.
- Phil Kearney’s blog post about the SMART framework.
- Qiu C, Lin JC, Shi JM, et al. Association Between Epidural Analgesia During Labor and Risk of Autism Spectrum Disorders in Offspring. JAMA Pediatr. 2020;174:1168-1175.
- Joint Statement. Labor Epidurals Do Not Cause Autism; Safe for Mothers and Infants, say Anesthesiology, Obstetrics, and Pediatric Medical Societies. American Society of Anesthesiologists. October 12, 2020.
- Wall-Wieler E, Bateman BT, Hanlon-Dearman A, Roos LL, Butwick AJ. Association of Epidural Labor Analgesia With Offspring Risk of Autism Spectrum Disorders. JAMA Pediatr. 2021;175:698-705.
- Christakis DA. More on epidurals and autism. JAMA Pediatrics. 2021; 175: 705.
- Mikkelsen AP, Greiber IK, Scheller NM, Lidegaard Ø. Association of Labor Epidural Analgesia With Autism Spectrum Disorder in Children. JAMA. 2021;326:1170–1177.
- Hanley GE, Bickford C, Ip A, et al. Association of Epidural Analgesia During Labor and Delivery With Autism Spectrum Disorder in Offspring. JAMA. 2021;326:1178-1185.
- Hegvik TA, Klungsøyr K, Kuja-Halkola R, et al. Labor epidural analgesia and subsequent risk of offspring autism spectrum disorder and attention-deficit/hyperactivity disorder: a cross-national cohort study of 4.5 million individuals and their siblings. Am J Obstet Gynecol. 2023;228:233.e1-233.e12. Epub 2022 Aug 13. Hu X, Wang B, Chen J, Han D, Wu J.
- Association Between Epidural Labor Analgesia and Autism Spectrum Disorder in Offspring: A Systematic Review and Meta-Analysis. J Pain Res;17:227-240.
Kristin and Regina’s online courses:Demystifying Data: A Modern Approach to Statistical Understanding
Clinical Trials: Design, Strategy, and Analysis
Medical Statistics Certificate Program
Epidemiology and Clinical Research Graduate Certificate Program
Programs that we teach in:
Epidemiology and Clinical Research Graduate Certificate Program
Find us on:
Kristin - LinkedIn & Twitter/X
Regina - LinkedIn & ReginaNuzzo.com
- (00:00) - Intro
- (01:40) - Why autism is hard to study
- (05:18) - The original 2020 study
- (11:38) - Results & hazard ratios
- (15:24) - Confounding & adjustment
- (26:32) - Criticism & plausibility
- (34:11) - Replications begin
- (44:38) - Converging evidence & meta-analysis
- (50:48) - What does it mean?
- (53:38) - Guest & wrap-up
Kapitel
Kapitel werden geladen …
Daylight Saving Time: Does springing forward cause heart attacks?
Every year we spring forward and lose an hour of sleep. But do we also lose a few heart cells? Some headlines claim that heart attacks spike by 24% after daylight saving time begins. In this episode we trace …
Shownotes
Every year we spring forward and lose an hour of sleep. But do we also lose a few heart cells? Some headlines claim that heart attacks spike by 24% after daylight saving time begins. In this episode we trace that number back to the research behind it—and what we find is more complicated than the headlines suggest. We examine a famous New England Journal of Medicine letter, a large international meta-analysis, and a massive modern U.S. registry study. Along the way we talk about incidence ratios, relative versus absolute risk, negative controls, and a haunting concept called harvesting. Plus: why bar charts are not for numerical data, why journalists love dramatic numbers, and how a bug collector helped invent daylight saving time.
Statistical topics
- Incidence ratios / incidence rates
- Meta-analysis
- Negative controls
- Relative risk vs absolute risk
- Statistical vs practical significance
- Statistical Sleuthing
Methodological morals- “A bump in time isn’t always a bump in total.”
- “If you already know the story you want to tell, you can always find a number to tell it.”
References- Bourke, India. “An obsessed insect hunter: The creepy-crawly origins of daylight saving.” BBC Future, March 31, 2024. https://www.bbc.com/future/article/20240308-how-first-suggestions-of-daylight-savings-time-was-inspired-by-insects
- Fox-Skelly, Jasmin. “How Daylight Saving Time Affects Your Health.” BBC Future, October 25, 2025. https://www.bbc.com/future/article/20251024-how-daylight-saving-time-affects-our-health
- Hurst A, Morfeld P, Lewis P, Erren TC. Daylight Saving Time Transitions and Risk of Heart Attack. Dtsch Arztebl Int. 2024;121(15):490-496. doi:10.3238/arztebl.m2024.0078
- Janszky I, Ljung R. Shifts to and from daylight saving time and incidence of myocardial infarction. N Engl J Med. 2008;359(18):1966-1968. doi:10.1056/NEJMc0807104
- Jiddou MR, Pica M, Boura J, Qu L, Franklin BA. Incidence of myocardial infarction with shifts to and from daylight savings time. Am J Cardiol. 2013;111(5):631-635. doi:10.1016/j.amjcard.2012.11.010
- Mellour, Richard. “The builder who changed how the world keeps time.” BBC Future, March 11, 2016. https://www.bbc.com/future/article/20160310-the-builder-who-changed-how-the-world-keeps-time
- Rymer JA, Li S, Chiswell K, et al. Daylight Savings Time and Acute Myocardial Infarction. JAMA Netw Open. 2025;8(9):e2530442. Published 2025 Sep 2. doi:10.1001/jamanetworkopen.2025.30442
- https://graph2table.com/
Kristin and Regina’s online courses:Demystifying Data: A Modern Approach to Statistical Understanding
Clinical Trials: Design, Strategy, and Analysis
Medical Statistics Certificate Program
Epidemiology and Clinical Research Graduate Certificate Program
Programs that we teach in:
Epidemiology and Clinical Research Graduate Certificate Program
Find us on:
Kristin - LinkedIn & Twitter/X
Regina - LinkedIn & ReginaNuzzo.com
- (00:00) - Intro
- (05:03) - Strange history of daylight saving time
- (16:06) - Swedish NEJM study
- (19:14) - Incidence ratios explained
- (22:13) - What the Swedish study actually found
- (31:11) - Absolute vs relative risk
- (34:27) - Harvesting effect
- (40:10) - 2024 Meta-analysis
- (45:37) - Large modern US study
- (55:23) - Where the “24% increase” came from
- (59:16) - Wrap-up
Kapitel
Kapitel werden geladen …
Marathon Performance: Does high-carb fueling work?
How many carbs do you need to run your best marathon? Recent headlines suggest that 120 grams per hour is the magic number. But what’s the science behind that claim? To find out, we dug into the study fueling …
Shownotes
How many carbs do you need to run your best marathon? Recent headlines suggest that 120 grams per hour is the magic number. But what’s the science behind that claim? To find out, we dug into the study fueling the hype — and were surprised by what we found. In this episode, we uncover numbers that mysteriously shift after peer review, figures that don’t match the text, and p-values that refuse to line up with their confidence intervals. Along the way, we swap bonking stories, revisit repeated-measures ANOVA, renew our antipathy for spreadsheets, and follow a trail of statistical termites to a surprisingly happy scientific ending.
Statistical topics- Article in press vs final publication
- Data management and workflow
- Multiple testing
- P-values and confidence intervals
- Repeated Measures ANOVA
- Statistical sleuthing
- Version control in research
- Within-person study design
Methodological morals- “Everyone makes statistical mistakes, not everyone fixes them.”
- “If the numbers aren't consistent, Excel is often part of the story.”
- “If a p-value doesn't survive the trip from text to figure, there's a problem.”
Statistical Sleuthing Extended Notes
References- Ravikanti S, Silang KG, Martyn HJ, et al. 13C-labelled glucose–fructose show greater exogenous and whole-body CHO oxidation and lower O2 cost of running at 120 versus 60 and 90 g·h−1 in elite male marathoners. J Appl Physiol. 2025;139:1581–95. (final version)
- Article in Press of J Appl Physiol. 2025;139:1581–95.
- graph2table AI data extraction from figures. Use the discount code normalcurves20 for 20% off!
- Bob Kempainen gutting out the win at the 1996 U.S. Olympic Marathon Trials.
- N=7 is a Dinner Party LinkedIn Post
- WebPlotDigitizer
- Holmer B. The new high-carb study that’s rocking the running world. Marathon Handbook. Dec 5, 2025.
- Ivy JL, Miller W, Dover V, et al. Endurance improved by ingestion of a glucose polymer supplement. Med Sci Sports Exerc. 1983; 15:466–471.
- Coyle EF, Coggan AR, Hemmert MK, et al. Muscle glycogen utilization during prolonged strenuous exercise when fed carbohydrate. J Appl Physiol. 1986; 61:165–172.
- Coggan AR, Coyle EF. Reversal of fatigue during prolonged exercise by carbohydrate infusion or ingestion. J Appl Physiol. 1987; 63:2388–2395.
- Below PR, Mora-Rodríguez R, González-Alonso J, et al. Fluid and carbohydrate ingestion independently improve performance during 1 h of intense exercise. Med Sci Sports Exerc. 1995; 27:200–210.
- American College of Sports Medicine. Position stand: Nutrition and athletic performance. Med Sci Sports Exerc. 1996.
- Jeukendrup AE, Jentjens R. Oxidation of carbohydrate feedings during prolonged exercise: current thoughts, guidelines and directions for future research. Sports Med. 2000; 29:407–424.
- Currell K, Jeukendrup AE. Superior endurance performance with ingestion of multiple transportable carbohydrates. Med Sci Sports Exerc. 2008; 40:275–281.
- American Dietetic Association, Dietitians of Canada, American College of Sports Medicine, et al. Nutrition and athletic performance. Med Sci Sports Exerc. 2009; 41:709–731.
- Triplett D, Doyle JA, Rupp JC, et al. An isocaloric glucose–fructose beverage's effect on simulated 100-km cycling performance compared with a glucose-only beverage. Int J Sport Nutr Exerc Metab. 2010; 20:122–131.
- Stellingwerff T, Cox GR. Systematic review: carbohydrate supplementation on exercise performance or capacity of varying durations. Appl Physiol Nutr Metab. 2014; 39:998–1011.
- Thomas DT, Erdman KA, Burke LM. Position of the Academy of Nutrition and Dietetics, Dietitians of Canada, and the American College of Sports Medicine: Nutrition and athletic performance. Med Sci Sports Exerc. 2016.
- King AJ, O’Hara JP, Morrison DJ, et al. Carbohydrate dose influences liver and muscle glycogen oxidation and performance during prolonged exercise. Physiol Rep. 2018; 6:e13555.
- Urdampilleta A, Mielgo-Ayuso J, Martínez-Sanz JM, et al. Effects of 120 vs 90 g·h⁻¹ carbohydrate intake during a mountain marathon on neuromuscular function and high-intensity run capacity recovery. Nutrients. 2020; 12:2099.
- Podlogar T, Bescós R, Wallis GA, et al. Increased exogenous but unaltered endogenous carbohydrate oxidation with 120 vs 90 g·h⁻¹ carbohydrate ingestion during prolonged endurance exercise. Front Nutr. 2022; 9:936691.
- Smith JW, Pascoe DD, Passe DH, et al. Curvilinear dose-response relationship of carbohydrate (0–120 g·h⁻¹) and performance. Med Sci Sports Exerc. 2013; 45:336–341.
- Lukasiewicz C, Vandiver KJ, Albert ED, et al. Assessing exogenous carbohydrate intake needed to optimize human endurance performance across sex: insights from modeling runners pursuing a sub-2-hour marathon. J Appl Physiol. 2024.
Kristin and Regina’s online courses:Demystifying Data: A Modern Approach to Statistical Understanding
Clinical Trials: Design, Strategy, and Analysis
...Kapitel
Kapitel werden geladen …
Falling in Love: Do the 36 Questions Actually Work?
Can a list of questions really make two strangers fall in love? In 2015, a viral New York Times Modern Love column claimed psychologists had discovered a formula for love: 36 increasingly personal questions …
Shownotes
Can a list of questions really make two strangers fall in love? In 2015, a viral New York Times Modern Love column claimed psychologists had discovered a formula for love: 36 increasingly personal questions, plus four minutes of eye contact. Millions of people tried it. There was even an app. But when we followed the citation trail back to the science, the story started to unravel. In this episode, we crack open the 1997 study behind the “36 Questions,” unearth a forgotten pilot study with a different (and sexier) protocol, and track down the real origin of the eye-gazing task. Along the way, we break down why control groups matter, why scale midpoints mislead, and why group averages aren’t people. We also try the questions on each other—purely for science, of course—and ask the nerdiest Valentine’s Day question of all: can a list of questions really make anyone fall in love?
Statistical topics- Control groups
- Correlated observations
- Group averages vs individual inference
- Pilot studies
- Reference distributions
- Scale interpretation
- Units of observation
Methodological morals- “Before you repeat a scientific claim, follow it back to the original study and read it carefully.”
- “You can slice the data into subgroups all you want, but that doesn't magically give you a control group. It gives you meaningless results.”
Our version of the “40 Questions” app!
References
- Aron, A., Aron, E.N., Melinat, E. and Vallone, R., 1991. Experimentally induced closeness, ego identity, and the opportunity to say no. In Conference of the International Network on Personal Relationships, Normal, IL.
- Aron, A., Melinat, E., Aron, E.N., Vallone, R.D. and Bator, R.J., 1997. The experimental generation of interpersonal closeness: A procedure and some preliminary findings. Personality and social psychology bulletin, 23(4), pp.363-377.
- Catron, Mandy L. To fall in love with anyone, do this. New York Times. January 11, 2015. https://www.nytimes.com/2015/01/11/style/modern-love-to-fall-in-love-with-anyone-do-this.html
- Catron, M.L., 2017. How to fall in love with anyone: a memoir in essays. Simon and Schuster.
- Jones, Daniel. The 36 Questions That Lead to Love. New York Times. January 9, 2015. https://www.nytimes.com/2015/01/09/style/no-37-big-wedding-or-small.html
- Kashdan, T.B. and Wenzel, A., 2005. A transactional approach to social anxiety and the genesis of interpersonal closeness: Self, partner, and social context. Behavior Therapy, 36(4), pp.335-346.
- Lee, Anna G. Long After ‘36 Questions,’ Finally Asking a Bigger One. New York Times. May 16, 2025. https://www.nytimes.com/2015/01/09/style/no-37-big-wedding-or-small.html
- Sprecher, S., 2021. Closeness and other affiliative outcomes generated from the Fast Friends procedure: A comparison with a small-talk task and unstructured self-disclosure and the moderating role of mode of communication. Journal of Social and Personal Relationships, 38(5), pp.1452-1471.
- Vacharkulksemsuk T, Fredrickson BL. Strangers in sync: Achieving embodied rapport through shared movements. J Exp Soc Psychol. 2012;48(1):399-402. doi:10.1016/j.jesp.2011.07.015
- Mandy Len Catron’s TEDx talk: https://www.youtube.com/watch?v=v8Yo-PXN7UA
- Ivan Vendrov’s Twitter/X post about his exchange with Arthur Aron: https://x.com/IvanVendrov/status/1611809736823013377/photo/1
- https://www.scientificamerican.com/podcast/episode/love-and-the-brain-part-1-the-36-questions-revisited/
- Our version of the “40 Questions” app: https://www.normalcurves.com/questions-to-fall-in-love/
Kristin and Regina’s online courses:Demystifying Data: A Modern Approach to Statistical Understanding
Clinical Trials: Design, Strategy, and Analysis
Medical Statistics Certificate Program
Epidemiology and Clinical Research Graduate Certificate Program
Programs that we teach in:
Epidemiology and Clinical Research Graduate Certificate Program
Find us on:
Kristin - LinkedIn & Twitter/X
Regina - LinkedIn & ReginaNuzzo.com
- (00:00) - Intro
- (04:42) - Viral NYT Modern Love essay’s cultural influence
- (09:32) - Science behind the 36 questions
- (15:07) - The 1997 paper myth busting
- (18:52) - Sleuthing the pilot study
- (29:44) - What did the 1997 paper actually show
- (41:24) - Discussion section
- (49:58) - Did it replicate
- (56:47) - Wrap up
Kapitel
Kapitel werden geladen …
Bonus: Pheromones with commentary
While we’re on a short break between seasons, we’re revisiting some of our favorite episodes from Season 1. This week, we’re re-releasing our debut episode on pheromones and sexy sweat, with some added …
Shownotes
While we’re on a short break between seasons, we’re revisiting some of our favorite episodes from Season 1. This week, we’re re-releasing our debut episode on pheromones and sexy sweat, with some added commentary up front..
Sweaty t-shirt dating parties, sex pheromone dating sites, choosing your dating partner by sniffing them up — wacko fringe fads or evidence-based mating strategies? And what does your armpit stain have to do with your kids’ immune systems, or hormonal contraceptive pills, or divorce rates?
In this episode, we reach back into the 1990s and revisit the scientific paper that started it all: The Sweaty T-Shirt Study. They bring a sharp eye and open mind, critically examining the study and following the line of research to today. Along the way, they encounter interesting statistical topics – including correlated observations, within-person study design, and bar chart blasphemy – with a short, surprising detour into Neanderthal sex.
Statistical topics- Bar charts
- Correlated observations
- Cherry-picking
- Data and methodological transparency
- Multiple testing
- Post-hoc analyses
- Unit of observation / unit of anaysis
- Within-person study design
Methodological morals“Repeat after me: Bar charts are not for numerical data.”
“Those who ignore dependencies in their data are destined for flawed conclusions.”
References- Nuzzo, R. Ah, Love at first whiff. Los Angeles Times. May 19, 2008.
- Papamarko, S. Pheromone parties attempt to match singles by scent. Yahoo!life. April 12, 2012.
- Sainani, K. Stone Age Gene Swap. Stanford Magazine. November/December 2011.
- Aldhous, P. Darling, You Smell Wonderfully Different. New Scientist. 6 May 1995.
- Wedekind C, Seebeck T, Bettens F, Paepke AJ. MHC-dependent mate preferences in humans. Proc Biol Sci. 1995; 260(1359):245-249. doi:10.1098/rspb.1995.0087
- Hedrick P, Loeschcke V. MHC and mate selection in humans?. Trends Ecol Evol. 1996;11(1):24. doi:10.1016/0169-5347(96)80237-0
- Wedekind C, Seebeck T. Reply from C. Wedekind and T. Seebeck. Trends Ecol Evol. 1996;11(1):24-25. doi:10.1016/0169-5347(96)81061-5
- Wedekind C, Füri S. Body odour preferences in men and women: do they aim for specific MHC combinations or simply heterozygosity?. Proc Biol Sci. 1997;264(1387):1471-1479. doi:10.1098/rspb.1997.0204
- Havlíček J, Winternitz J, Roberts SC. Major histocompatibility complex-associated odour preferences and human mate choice: near and far horizons. Philos Trans R Soc Lond B Biol Sci. 2020;375(1800):20190260. doi:10.1098/rstb.2019.0260
Kristin and Regina’s online courses:Demystifying Data: A Modern Approach to Statistical Understanding
Clinical Trials: Design, Strategy, and Analysis
Medical Statistics Certificate Program
Epidemiology and Clinical Research Graduate Certificate Program
Programs that we teach in:
Epidemiology and Clinical Research Graduate Certificate Program
Find us on:
Kristin - LinkedIn & Twitter/X
Regina - LinkedIn & ReginaNuzzo.com
- (00:00) - Introduction
- (00:27) - Pheromone Dating Parties
- (00:57) - Pheromone Dating Sites and Genetic Matching
- (00:47) - The Science of HLA Genes and Mate Selection
- (00:08) - Breaking Down the Original Sweaty T-Shirt Study
- (00:08) - Study Design Flaws and Data Transparency Issues
- (00:31) - Statistical Flaws: Correlated Observations Explained
- (00:22) - Analyzing the Study's Questionable Results
- (00:18) - The Pill's Influence on Scent Preferences
- (00:26) - Overstated Conclusions and Wandering Discussions
- (00:09) - Media Reactions and the Study’s Public Impact
- (00:22) - Other Studies and their results
- (00:01) - Conclusion
Kapitel
Kapitel werden geladen …
Bonus: Sugar Sag with Commentary
While we’re on a short break between seasons, we’re revisiting some of our favorite episodes from Season 1. This week, we’re re-releasing our exploration of how your diet can affect your skin – now with added …
Shownotes
While we’re on a short break between seasons, we’re revisiting some of our favorite episodes from Season 1. This week, we’re re-releasing our exploration of how your diet can affect your skin – now with added commentary!
Wrinkles and sagging skin—just normal aging, or can you blame your sweet tooth? We dive into “sugar sag,” exploring how sugar, processed foods, and even your crispy breakfast toast might be making you look older than if you’d said no to chocolate cake and yes to broccoli. Along the way, we encounter statistical adjustment, training and test data sets, what we call “references to nowhere,” plus some cadavers and collagen. Ever heard of an AGE reader? Find out how this tool might offer a sneak peek at your date’s age—and maybe even a clue about his… um… “performance.”
Statistical topics- Confounding
- Correlation vs causation
- Measurement error / proxy variables
- Overfitting
- Plagiarism
- Proper citing practices
- References to nowhere
- Statistical adjustment
- Training and test sets
Methodologic morals- “When you plagiarize, you steal the errors too.”
- “Overdone statistical adjustment is like overdone photo filters–at a certain point it’s just laughable.”
Citations
Collagen turnover:
- Verzijl N, DeGroot J, Thorpe SR, et al.Effect of Collagen Turnover on the Accumulation of Advanced Glycation End Products. JBC. 2000;275:39027-31.
Cadaver study:
- Hamlin CR, Kohn RR, Luschin JH. Apparent Accelerated Aging of Human Collagen in Diabetes Mellitus. Diabetes. 1975; 24: 902–904.
Studies of AGEs and diabetes and health:
- Monnier VM, Cerami A. Nonenzymatic browning in vivo: possible process for aging of long-lived proteins. Science. 1981;211:491-3.
- Brownlee M, Vlassara H, Cerami A. Nonenzymatic glycosylation and the pathogenesis of diabetic complications. Ann Intern Med. 1984;101:527-37.
- Monnier VM, Vishwanath V, Frank KE, et al. Relation between Complications of Type I Diabetes Mellitus and Collagen-Linked Fluorescence. N Engl J Med. 1986;314:403-408.
- Monnier VM, Sell DR, Abdul-Karim FW, et al. Collagen browning and cross-linking are increased in chronic experimental hyperglycemia. Relevance to diabetes and aging. Diabetes. 1988;37:867-72.
- Monnier VM, Bautista O, Kenny D, et al. Skin collagen glycation, glycoxidation, and crosslinking are lower in subjects with long-term intensive versus conventional therapy of type 1 diabetes: relevance of glycated collagen products versus HbA1c as markers of diabetic complications. Diabetes 1999; 48: 870–80.
- Genuth S, Sun W, Cleary P, et al. Glycation and carboxymethyllysine levels in skin collagen predict the risk of future 10-year progression of diabetic retinopathy and nephropathy in the diabetes control and complications trial and epidemiology of diabetes interventions and complications participants with type 1 diabetes. Diabetes. 2005;54:3103-11.
- van Waateringe RP, Slagter SN, van Beek AP, et al. Skin autofluorescence, a non-invasive biomarker for advanced glycation end products, is associated with the metabolic syndrome and its individual components. Diabetol Metab Syndr. 2017;9:42.
- Kouidrat Y, Zaitouni A, Amad A, et al. Skin autofluorescence (a marker for advanced glycation end products) and erectile dysfunction in diabetes. J Diabetes Complications. 2017;3:108-113.
- Fujita N, Ishida M, Iwane T, et al. Association between Advanced Glycation End-Products, Carotenoids, and Severe Erectile Dysfunction. World J Mens Health. 2023;41:701-11.
- Uruska A, Gandecka A, Araszkiewicz A, et al. Accumulation of advanced glycation end products in the skin is accelerated in relation to insulin resistance in people with Type 1 diabetes mellitus. Diabet Med. 2019;36:620-625.
- Boersma HE, Smit AJ, Paterson AD, et al. Skin autofluorescence and cause-specific mortality in a population-based cohort. Sci Rep 2024;14:19967.
Review article with conflicts of interest:
- Draelos ZD. Sugar Sag: What Is Skin Glycation and How Do You Combat It? J Drugs Dermatol. 2024; 23:s5-10.
Clinical study on AGE interrupter cream:
- Draelos ZD, Yatskayer M, Raab S, Oresajo C. An evaluation of the effect of a topical product containing C-xyloside and blueberry extract on the appearance of type II diabetic skin. J Cosmet Dermatol. 2009;8:147-51.
Our citation trail:
- 2023 review article: Zgutka K, Tkacz M, Tomasiak, et al. A Role for Advanced Glycation End Products in Molecular Ageing. Int J Mol Sci. 2023; 24: 9881. Sentence: “Interestingly, strict control of blood sugar for 4 months reduced the production of glycosylated collagen by 25%, and low-sugar food prepared by boiling could also reduce the production of AGEs [152].”
- Reference 152 is a review article: Cao C, Xiao Z, Wu Y, et al. Diet and Skin Aging-From the Perspective of Food Nutrition. Nutrients. 2020;12:870. Sentence: “However, strict control of blood sugar for four months can reduce the production of glycosylated collagen by 25%, and low-sugar food prepared by boiling can also reduce the production of AGEs [93–95].”
- Reference 93 is a review article: Nguyen HP, Katta R. Sugar sag: Glycation and the role of diet in aging skin. Skin Ther Lett. 2015; 20: 1–5. Sentence: “Tight glycemic control over a 4-month period can result in a reduction of glycated collagen formation by 25%.37,38”
- Reference 94 and 38 is a review article: Draelos ZD. Aging skin: the role of diet: facts and controversies. Clin Dermatol. 2013;31:701-6. Sentence: “Tighter glycemic control can reduce glycated collagen by 25% in 4 months.” No citation given. ...
Kapitel
Kapitel werden geladen …
Bonus: Vitamin D Part 1 with commentary
While we’re on a short break between seasons, we’re revisiting some of our favorite episodes from Season 1. This week, we’re re-releasing our deep dive into vitamin D and the origins of the so-called deficiency …
Shownotes
While we’re on a short break between seasons, we’re revisiting some of our favorite episodes from Season 1. This week, we’re re-releasing our deep dive into vitamin D and the origins of the so-called deficiency epidemic, with added commentary.
Is America really facing an epidemic of vitamin D deficiency? While this claim is widely believed, the story behind it is packed with twists, turns, and some pesky statistical cockroaches. In this episode, we’ll dive into a study on Hawaiian surfers, expose how shifting goalposts can create an epidemic, tackle dueling medical guidelines, and flex our statistical sleuthing skills. By the end, you might wonder if the real deficiency lies in the data.
Statistical topics
- conflicts of interest
- cut points and thresholds
- dichotomization
- incentives in science
- measurement and classification
- normal distribution
- researcher biases
- standard deviation
- statistical sleuthing
Methodologic morals
- “Arbitrary thresholds make for arbitrary diseases.”
- “Statistical errors are like cockroaches: Where there’s one, there’s many.”
Note that all blood vitamin D levels discussed in the podcast are 25-hydroxyvitamin D levels given in units of ng/ml. To convert from ng/ml to nmol/L, use the formula: nmol/L=2.5*ng/ml. For example, a vitamin D level of 30 ng/mL corresponds to 75 nmol/L.
Citations
Dr. Rhonda Patrick: Micronutrients for Health & Longevity. Huberman Lab Podcast. May 1, 2022Noh CK, Lee MJ, Kim BK, et al. A Case of Nutritional Osteomalacia in Young Adult Male. J Bone Metab. 2013; 20:51-55.
Binkley N, Novotny R, Krueger D, et al. Low vitamin D status despite abundant sun exposure. J Clin Endocrinol Metab. 2007;92:2130-5.
Malabanan A, Veronikis IE, Holick MF. Redefining Vitamin D Insufficiency. Lancet. 1998;351:805-6.
Dawson-Hughes B, Heaney RP, Holick MF, et al. Estimates of optimal vitamin D status. Osteoporos Int. 2005;16:713-6.
Holick MF. Vitamin D deficiency. N Engl J Med. 2007;357:266-81.
Cui A, Xiao P, Ma Y, et al. Prevalence, trend, and predictor analyses of vitamin D deficiency in the US population, 2001-2018. Front Nutr. 2022;9:965376.
Ross AC, Manson JE, Abrams SA, et al. The 2011 report on dietary reference intakes for calcium and vitamin D from the Institute of Medicine: what clinicians need to know. J Clin Endocrinol Metab. 2011;96:53-8.
Holick MF, Binkley NC, Bischoff-Ferrari HA, et al. Evaluation, Treatment, and Prevention of Vitamin D Deficiency: an Endocrine Society Clinical Practice Guideline. J Clin Endocrinol Metab. 2011;96:1911-30.
Manson JE, Brannon PM, Rosen CJ, et al. Vitamin D deficiency-is there really a pandemic. N Engl J Med. 2016;375:1817-20.
Conti G, Chirico V, Lacquaniti A, et al. Vitamin D intoxication in two brothers: be careful with dietary supplements. J Pediatr Endocrinol Metab. 2014;27:763-7.
Holick, Michael, et al. The UV Advantage. Ibooks, 2004.
Holick, Michael F. The Vitamin D Solution: A 3-Step Strategy to Cure Our Most Common Health Problems. Penguin Publishing Group, 2011.
Szabo, Liz. Vitamin D, the Sunshine Supplement, Has Shadowy Money Behind It. The New York Times. August 18, 2018.
Lee JM, Smith JR, Philipp BL, Chen TC, Mathieu J, Holick MF. Vitamin D deficiency in a healthy group of mothers and newborn infants. Clin Pediatr. 2007;46:42-4.
Holick MF. Vitamin D deficiency: what a pain it is. Mayo Clin Proc. 2003;78:1457-9.
Passeri G, Pini G, Troiano L, et al. Low Vitamin D Status, High Bone Turnover, and Bone Fractures in Centenarians. J Clin Endocrinol Metab. 2003;88:5109-15.
Armstrong, David. The Child Abuse Contrarian. ProPublica. September 16, 2018.
Irwig MS, Kyinn M, Shefa MC. Financial Conflicts of Interest Among Authors of Endocrine Society Clinical Practice Guidelines. J Clin Endocrinol Metab. 2018;103:4333-38.
Demay MB, Pittas AG, Bikle DD, et al. Vitamin D for the Prevention of Disease: An Endocrine Society Clinical Practice Guideline. J Clin Endocrinol Metab. 2024;109:1907-47.
McCartney CR, McDonnell ME, Corrigan MD, et al. Vitamin D Insufficiency and Epistemic Humility: An Endocrine Society Guideline Communication. J Clin Endocrinol Metab. 2024; 109:1948–54.
See our detailed notes hereKristin and Regina’s online courses
Demystifying Data: A Modern Approach to Statistical UnderstandingClinical Trials: Design, Strategy, and Analysis
Medical Statistics Certificate Program
Epidemiology and Clinical Research Graduate Certificate Program
Programs that we teach in:
Epidemiology and Clinical Research Graduate Certificate Program
Find us on:
Kristin - LinkedIn & Twitter/X
Regina - LinkedIn...
Kapitel
Kapitel werden geladen …
The Batman Effect: Do weird surprises make people nicer?
DescriptionNobody expects Batman—but when he shows up in a crowded subway car, are people suddenly more likely to help a passenger in need? This week on Normal Curves, we unpack a recent quasi-experimental …
Shownotes
Description
Nobody expects Batman—but when he shows up in a crowded subway car, are people suddenly more likely to help a passenger in need? This week on Normal Curves, we unpack a recent quasi-experimental field study involving a caped superhero costume, a prosthetic pregnancy belly, and some puzzled Italian commuters. Along the way, we demystify three common ways of describing effects for binary outcomes—risk differences, risk ratios, and odds ratios—and explain what they actually mean in plain language. We also do some statistical sleuthing, uncover a major problem hiding in the paper’s numbers, and debate what really counts as an effective Batman outfit.
Statistical topics- absolute vs relative effects
- binary outcomes
- coding errors
- data errors and quality control
- effect size interpretation
- field experiments
- odds
- odds ratios
- percentage differences
- quasi-experimental studies
- risk differences
- risk ratios
- statistical sleuthing
Methodological morals
- “We love an uncluttered paper, but when it's missing the basics, it's like an empty fridge. Clean, yes, but dinner is not happening.”
- “Before you make a fancy model, make sure the numbers in the table in the text match.”
References- Pagnini F, Grosso F, Cavalera C, et al. Unexpected events and prosocial behavior: the Batman effect. Npj Ment Health Res. 2025;4(1):57. Published 2025 Nov 3. doi:10.1038/s44184-025-00171-5
- PubPeer. Comments on “Unexpected events and prosocial behavior: the Batman effect.” Accessed December 2025.
- Sainani KL. Understanding odds ratios. PM R. 2011;3(3):263-267. doi:10.1016/j.pmrj.2011.01.009
- Nuzzo RL. Communicating measures of relative risk in plain English. PM R. 2022;14(2):283-287. doi:10.1002/pmrj.12761
- Sainani KL. How statistics can mislead. Am J Public Health. 2012;102:e3-4.
Kristin and Regina’s online courses:Demystifying Data: A Modern Approach to Statistical Understanding
Clinical Trials: Design, Strategy, and Analysis
Medical Statistics Certificate Program
Epidemiology and Clinical Research Graduate Certificate Program
Programs that we teach in:
Epidemiology and Clinical Research Graduate Certificate Program
Find us on:
Kristin - LinkedIn & Twitter/X
Regina - LinkedIn & ReginaNuzzo.com
- (00:00) - Intro
- (03:42) - Why would Batman make people nicer?
- (07:33) - How they ran the experiment
- (17:06) - Did Batman save the day? Different ways to answer that
- (22:16) - What are odds and odds ratios?
- (29:16) - Where people get it wrong
- (34:08) - The plot twist: big numerical errors
- (40:36) - Did men or women give up their seat more often?
- (43:05) - Wrap-up and methodological morals
Kapitel
Kapitel werden geladen …
Holiday Survival Guide Part 2: The survey study edition
Does the temperature of your coffee six months ago really predict whether you feel gassy today? This week we dissect a new nutrition survey study on hot and cold beverage habits that claims to connect drink …
Shownotes
Does the temperature of your coffee six months ago really predict whether you feel gassy today? This week we dissect a new nutrition survey study on hot and cold beverage habits that claims to connect drink temperature with gut symptoms, anxiety, and more—despite relying on year-old memories and a blizzard of statistical tests. It’s the perfect case study for our Holiday Survival Guide Part 2, where we teach you how to talk with Uncle Joe at the dinner table about one of the most common—and most fraught—study designs in science: cross-sectional surveys. We walk through our easy checklist for making sense of results, show how recall bias and measurement error can skew the story, and reacquaint you with nonmonogamous Multiple-Testing Dude, who’s been very busy in this dataset. A friendly, practical guide to spotting when researchers are just torturing the data until it confesses.
Statistical topics- Confounding
- Cross-sectional studies
- False positives
- Measurement error
- Multiple testing
- PICOT / PIVOT framework
- Recall bias
- Research hypotheses
- Sample size and power
- Signal vs. noise
- SMART framework
- Statistical significance
- Subgroup analyses
- Survey design
- Transparency and trustworthiness
Methodological morals- “When your measurement starts with ‘think back to last winter’ you might as well use a random number generator.”
- “If the effect is only significant in certain subgroups in certain seasons for certain outcomes, it might just be a bad case of gas.”
References- Wu T, Doyle C, Ito J, et al. Cold Exposures in Relation to Dysmenorrhea among Asian and White Women. Int J Environ Res Public Health. 2023;21(1):56. Published 2023 Dec 30. doi:10.3390/ijerph21010056
- Wu T, Ramesh N, Doyle C, Hsu FC. Cold and hot consumption and health outcomes among US Asian and White populations. Br J Nutr. Published online September 18, 2025. doi:10.1017/S000711452510514X
Kristin and Regina’s online courses:Demystifying Data: A Modern Approach to Statistical Understanding
Clinical Trials: Design, Strategy, and Analysis
Medical Statistics Certificate Program
Epidemiology and Clinical Research Graduate Certificate Program
Programs that we teach in:
Epidemiology and Clinical Research Graduate Certificate Program
Find us on:
Kristin - LinkedIn & Twitter/X
Regina - LinkedIn & ReginaNuzzo.com
- (00:00) - Intro
- (04:36) - Did they have real research hypotheses?
- (10:29) - Observational or randomized experiment?
- (19:25) - PICOT and PIVOT
- (25:36) - Memory problems
- (31:19) - Five outcomes and measurement problems therein
- (36:12) - SMART
- (40:31) - Multiple Testing Dude is having a great time
- (51:17) - How big is the effect?
- (57:47) - Wrap-up and Irish Coffee rating scale
Kapitel
Kapitel werden geladen …
Holiday Survival Guide: How to talk about scientific studies around the dinner table
Does a little alcohol really make you speak a foreign language better? This week we unpack a quirky randomized trial that tested Dutch pronunciation after a modest buzz—and came to the opposite conclusion the …
Shownotes
Does a little alcohol really make you speak a foreign language better? This week we unpack a quirky randomized trial that tested Dutch pronunciation after a modest buzz—and came to the opposite conclusion the researchers expected. We use it as the perfect holiday case study: instead of arguing with Uncle Joe at the dinner table, we’ll show you how to pull apart a scientific headline using a friendly, practical checklist anyone can learn. Along the way we stress-test the study’s claims, take a quick detour into what a .04% buzz actually looks like, and run our own before-and-after experiment with two brave science journalists at the ScienceWriters2025 conference in Chicago. A holiday survival guide with vodka tonics, statistical sleuthing, and a few surprisingly smooth French phrases.
Statistical topics
- Alternative explanations
- Arithmetic consistency / GRIM test
- Blinding
- Effect size / magnitude
- Generalizability / external validity
- Observational studies vs. experiments
- Outcome measurement
- PICOT framework
- Placebo and expectancy effects
- Primary outcomes / pre-specification
- Randomized controlled trials
- Research hypotheses
- Sample size
- SMART framework
- Statistical significance (signal vs. noise)
- Transparency and trustworthiness
Methodological morals
- “You don't need a PhD to read a study. Just remember, PICOT and SMART.”
- “A decimal point can mean the difference between life and death. Details matter.”
References
- Renner F, Kersbergen I, Field M, Werthmann J. Dutch courage? Effects of acute alcohol consumption on self-ratings and observer ratings of foreign language skills. J Psychopharmacol. 2018;32(1):116-122. doi:10.1177/0269881117735687
Kristin and Regina’s online courses:Demystifying Data: A Modern Approach to Statistical Understanding
Clinical Trials: Design, Strategy, and Analysis
Medical Statistics Certificate Program
Epidemiology and Clinical Research Graduate Certificate Program
Programs that we teach in:
Epidemiology and Clinical Research Graduate Certificate Program
Find us on:
Kristin - LinkedIn & Twitter/X
Regina - LinkedIn & ReginaNuzzo.com
- (00:00) - Intro
- (03:30) - Uncle Joe and the question of alcohol
- (07:20) - Randomized controlled trial
- (10:10) - PICOT mnemonic
- (15:43) - Just how drunk?
- (21:41) - Boring non-placeb
- (32:29) - Kristin’s SMART mnemonic
- (38:15) - How big of an effect?
- (49:29) - Two science journalists walk into a bar
- (55:43) - Martini scale and wrap-up
Kapitel
Kapitel werden geladen …
Shingles Shot and Dementia: Could one vaccine protect your brain?
What do chickenpox and shingles have to do with your brain? This week, we dig into two 2025 headline-grabbing studies that link the shingles shot to lower dementia rates. We start in Wales, where a birthday …
Shownotes
What do chickenpox and shingles have to do with your brain? This week, we dig into two 2025 headline-grabbing studies that link the shingles shot to lower dementia rates. We start in Wales, where a birthday cutoff turned into the perfect natural experiment, and end in the U.S. with a multi-million-person megastudy. Featuring bias-variance Goldilockses, Fozzy-the-Bear regression discontinuities, a Barbie-versus-Oppenheimer showdown for propensity scores – and the hottest rebrand of inverse-probability weighting you’ll ever hear.
Statistical topics- Absolute vs. relative risk
- Bias–variance tradeoff
- Causal inference
- Censoring
- Confounding
- Fuzzy regression discontinuity design
- Healthy-user bias
- Inverse probability of treatment weighting (IPTW)
- Longitudinal study
- Natural experiment
- Negative controls
- Optimal bandwidth
- Propensity scores
- Selection bias
- Subgroup analysis
- Triangular kernel weights
Methodological morals- “Propensity scores are the lipstick you put on observational pigs.”
- “Natural experiments are a hot flirtation date with causality.”
References- Eyting M, Xie M, Michalik F, Heß S, Chung S, Geldsetzer P. A natural experiment on the effect of herpes zoster vaccination on dementia. Nature. 2025 May;641(8062):438-446. doi: 10.1038/s41586-025-08800-x. Epub 2025 Apr 2. PMID: 40175543; PMCID: PMC12058522.
- Polisky V, Littmann M, Triastcyn A, et al. Varicella-zoster virus reactivation and the risk of dementia. Nat Med. Published online October 6, 2025. doi:10.1038/s41591-025-03972-5
- Sainani KL. Propensity scores: uses and limitations. PM&R 2012; 4:693-97.
Kristin and Regina’s online courses:Demystifying Data: A Modern Approach to Statistical Understanding
Clinical Trials: Design, Strategy, and Analysis
Medical Statistics Certificate Program
Epidemiology and Clinical Research Graduate Certificate Program
Programs that we teach in:
Epidemiology and Clinical Research Graduate Certificate Program
Find us on:
Kristin - LinkedIn & Twitter/X
Regina - LinkedIn & ReginaNuzzo.com
- (00:00) - Intro and first gratuitous mention of sex
- (03:56) - What are shingles, chickenpox, and the vaccines against them?
- (12:30) - Fun facts about the varicella zoster and herpes viruses
- (17:16) - A natural experiment in Wales
- (21:10) - What is the Goldilocks optimal bandwidth?
- (25:33) - Fuzzy regression discontinuity design demystified
- (31:59) - Shingles vaccine vs dementia showdown
- (33:29) - Absolute risk reduction paradox
- (37:00) - Effects for men and women differ
- (39:48) - A giant longitudinal study
- (46:32) - Propensity scores demystified via Barbie and Oppenheimer
- (52:36) - Using propensity scores to make matches
- (56:49) - Inverse probability of treatment weighting demystified via more Barbenheimer
- (01:01:08) - Attempts to rename IPTW for TikTok
- (01:04:40) - Longitudinal study results
- (01:08:41) - Smooch ratings and methodological morals: pigs and hot dates
Kapitel
Kapitel werden geladen …
Scary Bridge Study: Can fear make you horny?
What if a haunted house makes your date look hotter? This week we dive into the infamous Scary Bridge Study — the 1970s classic that launched a thousand pop-psych takes on fear and lust. It’s the one with the …
Shownotes
What if a haunted house makes your date look hotter? This week we dive into the infamous Scary Bridge Study — the 1970s classic that launched a thousand pop-psych takes on fear and lust. It’s the one with the swaying bridge, pretty “research assistant,” and phone number scrawled on torn paper. The study became legend, but how sturdy were its stats? We retrace the design, redo the numbers, and see how many math errors it takes to sway a suspension bridge. Along the way we find an erotic-fiction writing exercise, Adventure Dudes choosing their own experimental groups, and snarky replicators who tried (and failed) to make fear sexy again. We wrap with what the latest research says about when fear really does boost attraction — and when it backfires spectacularly. A Halloween story of danger, desire, and unconscious sexual drive.
This episode has a video version! https://www.youtube.com/watch?v=2coWoS_3460
Statistical topics- Arithmetic checks
- Chi-square test
- Confounders
- GRIM test
- Inter-rater reliability
- Meta-analysis
- Negative control
- Randomization
- Replication
- Sample size
- Signal vs. noise
- Statistical sleuthing
- Subjective measurement
- T-test
Methodological morals
- “Those who don't verify their numbers dig their own statistical graves.”
- “Famous doesn't mean flawless.”
References- Brown, NJ, Heathers, JA. The GRIM test: A simple technique detects numerous anomalies in the reporting of results in psychology. Social Psychological and Personality Science. 2017; 8(4):363-369.
- Dutton DG, Aron AP. Some evidence for heightened sexual attraction under conditions of high anxiety. J Pers Soc Psychol. 1974;30(4):510-517. doi:10.1037/h0037031
- Foster CA, Witcher BS, Campbell WK, Green JD. Arousal and attraction: Evidence for automatic and controlled processes. J Pers Soc Psychol. 1998;74(1):86-101.
- Kenrick DT, Cialdini R, Linder D. Misattribution under fear-producing circumstances: Four failures to replicate. Pers Soc Psychol Bull. 1979;5(3):329-334.
- van der Zee T, Anaya J, Brown NJL. Statistical heartburn: an attempt to digest four pizza publications from the Cornell Food and Brand Lab. BMC Nutr. 2017;3:54. Published 2017 Jul 10. doi:10.1186/s40795-017-0167-x
- http://www.prepubmed.org/grim_test/
Kristin and Regina’s online courses:
Demystifying Data: A Modern Approach to Statistical Understanding
Clinical Trials: Design, Strategy, and Analysis
Medical Statistics Certificate Program
Epidemiology and Clinical Research Graduate Certificate Program
Programs that we teach in:
Epidemiology and Clinical Research Graduate Certificate Program
Find us on:
Kristin - LinkedIn & Twitter/X
Regina - LinkedIn & ReginaNuzzo.com
- (00:00) - Intro: Fear and Flirtation on a Suspension Bridge
- (05:40) - A Classic 1970s Experiment with No IRB to be Found
- (11:15) - Adventure Dudes Choose Their Own Bridge
- (17:00) - The Sexy Story Scale
- (22:20) - Cool Factor and the Negative Control
- (28:10) - Grim Reaper Math
- (35:45) - T-Tests, Chi-Squares, and Shaky Results
- (42:00) - Electric Shocks and Damsels in Distress
- (49:30) - Replications and Rejections
- (57:20) - Wrap-Up, Methodological Morals, and a New Sexy Rating Scale
Kapitel
Kapitel werden geladen …
Ultramarathons: Can vitamin D protect your bones?
Ultramarathoners push their bodies to the limit, but can a giant pre-race dose of vitamin D really keep their bones from breaking down? In this episode, we dig into a trial that tested this claim – and found a …
Shownotes
Ultramarathoners push their bodies to the limit, but can a giant pre-race dose of vitamin D really keep their bones from breaking down? In this episode, we dig into a trial that tested this claim – and found a statistical endurance event of its own: six highly interchangeable papers sliced from one small study. Expect missing runners, recycled figures, and a peer-review that reads like stand-up comedy, plus a quick lesson in using degrees of freedom as your statistical breadcrumbs.
Statistical topics- Data cleaning and validation
- Degrees of freedom
- Exploratory vs confirmatory analysis
- False positives and Type I error
- Intention-to-treat principle
- Multiple testing
- Open data and transparency
- P-hacking
- Salami slicing
- Parametric vs non-parametric tests
- Peer review quality
- Randomized controlled trials
- Research reproducibility
- Statistical sleuthing
Methodological morals
- “Degrees of freedom are the breadcrumbs in statistical sleuthing. They reveal the sample size even when the authors do not.”
- “Publishing the same study again and again with only the outcomes swapped is Mad Libs Science, better known as salami slicing.”
References- Boswell, Rachel. Pre-race vitamin D could do wonders for ultrarunners’ bone health, according to science. Runner’s World. September 25, 2025.
- Mieszkowski J, Stankiewicz B, Kochanowicz A, et al. Ultra-Marathon-Induced Increase in Serum Levels of Vitamin D Metabolites: A Double-Blind Randomized Controlled Trial. Nutrients. 2020;12(12):3629. Published 2020 Nov 25. doi:10.3390/nu12123629
- Mieszkowski J, Borkowska A, Stankiewicz B, et al. Single High-Dose Vitamin D Supplementation as an Approach for Reducing Ultramarathon-Induced Inflammation: A Double-Blind Randomized Controlled Trial. Nutrients. 2021;13(4):1280. Published 2021 Apr 13. doi:10.3390/nu13041280
- Mieszkowski J, Brzezińska P, Stankiewicz B, et al. Direct Effects of Vitamin D Supplementation on Ultramarathon-Induced Changes in Kynurenine Metabolism. Nutrients. 2022;14(21):4485. Published 2022 Oct 25. doi:10.3390/nu14214485
- Mieszkowski J, Brzezińska P, Stankiewicz B, et al. Vitamin D Supplementation Influences Ultramarathon-Induced Changes in Serum Amino Acid Levels, Tryptophan/Branched-Chain Amino Acid Ratio, and Arginine/Asymmetric Dimethylarginine Ratio. Nutrients. 2023;15(16):3536. Published 2023 Aug 11. doi:10.3390/nu15163536
- Stankiewicz B, Mieszkowski J, Kochanowicz A, et al. Effect of Single High-Dose Vitamin D3 Supplementation on Post-Ultra Mountain Running Heart Damage and Iron Metabolism Changes: A Double-Blind Randomized Controlled Trial. Nutrients. 2024;16(15):2479. Published 2024 Jul 31. doi:10.3390/nu16152479
- Stankiewicz B, Kochanowicz A, et al. Single high-dose vitamin D supplementation impacts ultramarathon-induced changes in serum levels of bone turnover markers: a double-blind randomized controlled trial. J Int Soc Sports Nutr. 2025 Dec;22(1):2561661. doi: 10.1080/15502783.2025.2561661.
Kristin and Regina’s online courses:
Demystifying Data: A Modern Approach to Statistical Understanding
Clinical Trials: Design, Strategy, and Analysis
Medical Statistics Certificate Program
Epidemiology and Clinical Research Graduate Certificate Program
Programs that we teach in:
Epidemiology and Clinical Research Graduate Certificate Program
Find us on:
Kristin - LinkedIn & Twitter/X
Regina - LinkedIn & ReginaNuzzo.com
00:00 Intro & claim of the episode
00:44 Runner’s World headline: Vitamin D for ultramarathoners
02:03 Kristin’s connection to running and vitamin D skepticism
03:32 Ultramarathon world—Regina’s stories and Death Valley race
06:29 What ultramarathons do to your bones
08:02 Boy story: four stress fractures in one race
10:00 Study design—40 male runners in Poland
11:33 Missing flow diagram and violated intention-to-treat
13:02 The intervention: 150,000 IU megadose
15:09 Blinding details and missing randomization info
17:13 Measuring bone biomarkers—no primary outcome specified
19:12 The wrong clinicaltrials.gov registration
20:35 Discovery of six papers from one dataset (salami slicing)
23:02 Why salami slicing misleads readers
25:42 Inconsistent reporting across papers
29:11 Changing inclusion criteria and sloppy methods
31:06 Typos, Polish notes, and misnumbered references
32:39 Peer review comedy gold—“Please define vitamin D”
36:06 Reviewer laziness and p-hacking admission
39:13 Results: implausible bone growth mid-race
41:16 Degrees of freedom sleuthing reveals hidden sample sizes
47:07 Open data? Kristin emails the authors
48:42 Lessons from Kristin’s own ultramarathon dataset
51:22 Fishing expeditions and misuse of parametric tests
53:07 Strength of evidence: one smooch each
54:44 Methodologic morals—Mad Libs Science & degrees of freedom breadcrumbs
56:12 Anyone can spot red flags—trust your eyes
57:34 Outro: skip the vitamin D shot before your next runP-Values: Are we using a flawed statistical tool?
P-values show up in almost every scientific paper, yet they’re one of the most misunderstood ideas in statistics. In this episode, we break from our usual journal-club format to unpack what a p-value really is …
Shownotes
P-values show up in almost every scientific paper, yet they’re one of the most misunderstood ideas in statistics. In this episode, we break from our usual journal-club format to unpack what a p-value really is, why researchers have fought about it for a century, and how that famous 0.05 cutoff became enshrined in science. Along the way, we share stories from our own papers—from a Nature feature that helped reshape the debate to a statistical sleuthing project that uncovered a faulty method in sports science. The result: a behind-the-scenes look at how one statistical tool has shaped the culture of science itself.
Statistical topics- Bayesian statistics
- Confidence intervals
- Effect size vs. statistical significance
- Fisher’s conception of p-values
- Frequentist perspective
- Magnitude-Based Inference (MBI)
- Multiple testing / multiple comparisons
- Neyman-Pearson hypothesis testing framework
- P-hacking
- Posterior probabilities
- Preregistration and registered reports
- Prior probabilities
- P-values
- Researcher degrees of freedom
- Significance thresholds (p < 0.05)
- Simulation-based inference
- Statistical power
- Statistical significance
- Transparency in research
- Type I error (false positive)
- Type II error (false negative)
- Winner’s Curse
Methodological morals
- “If p-values tell us the probability the null is true, then octopuses are psychic.”
- “Statistical tools don't fool us, blind faith in them does.”
References- Nuzzo R. Scientific method: statistical errors. Nature. 2014 Feb 13;506(7487):150-2. doi: 10.1038/506150a.
- Nuzzo, R., 2015. Scientists perturbed by loss of stat tools to sift research fudge from fact. Scientific American, pp.16-18.
- Nuzzo RL. The inverse fallacy and interpreting P values. PM&R. 2015 Mar;7(3):311-4. doi: 10.1016/j.pmrj.2015.02.011. Epub 2015 Feb 25.
- Nuzzo, R., 2015. Probability wars. New Scientist, 225(3012), pp.38-41.
- Sainani KL. Putting P values in perspective. PM&R. 2009 Sep;1(9):873-7. doi: 10.1016/j.pmrj.2009.07.003.
- Sainani KL. Clinical versus statistical significance. PM&R. 2012 Jun;4(6):442-5. doi: 10.1016/j.pmrj.2012.04.014.
- McLaughlin MJ, Sainani KL. Bonferroni, Holm, and Hochberg corrections: fun names, serious changes to p values. PM&R. 2014 Jun;6(6):544-6. doi: 10.1016/j.pmrj.2014.04.006. Epub 2014 Apr 22.
- Sainani KL. The Problem with "Magnitude-based Inference". Med Sci Sports Exerc. 2018 Oct;50(10):2166-2176. doi: 10.1249/MSS.0000000000001645.
- Sainani KL, Lohse KR, Jones PR, Vickers A. Magnitude-based Inference is not Bayesian and is not a valid method of inference. Scand J Med Sci Sports. 2019 Sep;29(9):1428-1436. doi: 10.1111/sms.13491.
- Lohse KR, Sainani KL, Taylor JA, Butson ML, Knight EJ, Vickers AJ. Systematic review of the use of "magnitude-based inference" in sports science and medicine. PLoS One. 2020 Jun 26;15(6):e0235318. doi: 10.1371/journal.pone.0235318.
- Wasserstein, R.L. and Lazar, N.A., 2016. The ASA statement on p-values: context, process, and purpose. The American Statistician, 70(2), pp.129-133.
Kristin and Regina’s online courses:Demystifying Data: A Modern Approach to Statistical Understanding
Clinical Trials: Design, Strategy, and Analysis
Medical Statistics Certificate Program
Epidemiology and Clinical Research Graduate Certificate Program
Programs that we teach in:
Epidemiology and Clinical Research Graduate Certificate Program
Find us on:
Kristin - LinkedIn & Twitter/X
Regina - LinkedIn & ReginaNuzzo.com
- (00:00) - Intro & claim of the episode
- (01:00) - Why p-values matter in science
- (02:44) - What is a p-value? (ESP guessing game)
- (06:47) - Big vs. small p-values (psychic octopus example)
- (08:29) - Significance thresholds and the 0.05 rule
- (09:00) - Regina’s Nature paper on p-values
- (11:32) - Misconceptions about p-values
- (13:18) - Fisher vs. Neyman-Pearson (history & feud)
- (16:26) - Botox analogy and type I vs. type II errors
- (19:41) - Dating app analogies for false positives/negatives
- (22:02) - How the 0.05 cutoff got enshrined
- (23:46) - Misinterpretations: statistical vs. practical significance
- (25:22) - Effect size, sample size, and “statistically discernible”
- (25:51) - P-hacking and researcher degrees of freedom
- (28:52) - Transparency, preregistration, and open science
- (29:58) - The 0.05 cutoff trap (p = 0.049 vs 0.051)
- (30:24) - The biggest misinterpretation: what p-values actually mean
- (32:35) - Paul the psychic octopus (worked example)
- (35:05) - Why Bayesian statistics differ
- (38:55) - Why aren’t we all Bayesian? (probability wars)
- (40:11) - The ASA p-value statement (behind the scenes)
- (42:22) - Key principles from the ASA white paper
- (43:21) - Wrapping up Regina’s paper
- (44:39) - Kristin’s paper on sports science (MBI)
- (47:16) - What MBI is and how it spread
- (49:49) - How Kristin got pulled in (Christie Aschwanden & FiveThirtyEight)
- (53:11) - Critiques of MBI and “Bayesian monster” rebuttal
- (55:20) - Spreadsheet autopsies (Welsh & Knight)
- (57:11) - Cherry juice example (why MBI misleads)
- (59:28) - Rebuttals and smoke & mirrors from MBI advocates
- (01:02:01) - Winner’s Curse and small samples
- (01:02:44) - Twitter fights & “establishment statistician”
- (01:05:02) - Cult-like following & Matrix red pill analogy
- (01:07:12) - Wrap-up
Kapitel
Kapitel werden geladen …
Hier stehen die neuesten 40 von 47 Folgen. Ältere Folgen findest du im Feed des Podcasts.
Ähnliche Podcasts
Mehr Gesellschaft und Kultur-Podcasts
Ist das dein Podcast?
Bestätige es mit einem Eintrag in deinem Feed: Du bekommst hier einen Code, trägst ihn als podcast:txt in den Kanal deines Feeds ein, und wir prüfen ihn. Danach steht dein Podcast als „verifiziert“ bei SoundNode.