Sean Harrison, PhD
@sean-h.bsky.social
Pinned
Sean Harrison, PhD
@sean-h.bsky.social
· Jun 14
I was saving this for an article, but I want something to refer to when I talk about how I think the best way to fix peer-review is to abolish journals.
Long-ish thread, and I'm talking from the perspective of epidemiology, where crappy studies can cause harm to individuals and populations.
1/n
Long-ish thread, and I'm talking from the perspective of epidemiology, where crappy studies can cause harm to individuals and populations.
1/n
January 20, 2026 at 1:27 PM
Unpopular opinion (from someone who works in evidence synthesis):
Meta-analyses shouldn't be at the top of the triangle, the should be alongside RCTs.
The best RCT will have better information quality than the best meta-analysis (on the same topic), unless the meta-analysis only includes one RCT.
Meta-analyses shouldn't be at the top of the triangle, the should be alongside RCTs.
The best RCT will have better information quality than the best meta-analysis (on the same topic), unless the meta-analysis only includes one RCT.
Everyone please red this and then vigorously attack @timpmorris.bsky.social 😜
tpmorris.substack.com/p/how-to-rea...
tpmorris.substack.com/p/how-to-rea...
How to read ‘evidence pyramids’
To get past the pointless bit of the arguments
tpmorris.substack.com
January 16, 2026 at 1:59 PM
Unpopular opinion (from someone who works in evidence synthesis):
Meta-analyses shouldn't be at the top of the triangle, the should be alongside RCTs.
The best RCT will have better information quality than the best meta-analysis (on the same topic), unless the meta-analysis only includes one RCT.
Meta-analyses shouldn't be at the top of the triangle, the should be alongside RCTs.
The best RCT will have better information quality than the best meta-analysis (on the same topic), unless the meta-analysis only includes one RCT.
The Secret Diary of The Children's Crusade
I thought it might be nice to give away my last 2 signed hardbacks of my novel 1983, with two lovely linocut hares by my mum.
All you have to do to enter is repost this (& leave a reply).
The book's been called "Stranger Things rewritten by Kurt Vonnegut and Sue Townsend”.
Extract in the replies.
All you have to do to enter is repost this (& leave a reply).
The book's been called "Stranger Things rewritten by Kurt Vonnegut and Sue Townsend”.
Extract in the replies.
January 9, 2026 at 1:53 PM
The Secret Diary of The Children's Crusade
For the past year, I've been working on some evidence reviews for the role of AI in public health, and am a little sad I can't include the following quote in any of them:
January 9, 2026 at 1:45 PM
For the past year, I've been working on some evidence reviews for the role of AI in public health, and am a little sad I can't include the following quote in any of them:
Scrubs has to be up there.
Ignoring medschool and the new series...
Ignoring medschool and the new series...
Given that apparently the Stranger Things finale was meh (idk, didn't watch it, just the scuttlebutt) and we're not that far removed from the disastrous GOT finale that retroactively made everyone have never cared about the show:
What's the *best* ending to a show you've ever seen? Quote/reply etc
What's the *best* ending to a show you've ever seen? Quote/reply etc
January 7, 2026 at 9:38 AM
Scrubs has to be up there.
Ignoring medschool and the new series...
Ignoring medschool and the new series...
When I read through parts of the Cass review when it came out, I thought the same as Chris here.
Big difference between "absence of evidence" and "evidence of absence", and even academics sometimes struggle with the difference.
If there's absence of evidence, you need more evidence!
Good letter.
Big difference between "absence of evidence" and "evidence of absence", and even academics sometimes struggle with the difference.
If there's absence of evidence, you need more evidence!
Good letter.
"The Cass review did not find puberty blockers unsafe; it found the evidence base to be weak." Letter in today's Times
December 16, 2025 at 2:47 PM
When I read through parts of the Cass review when it came out, I thought the same as Chris here.
Big difference between "absence of evidence" and "evidence of absence", and even academics sometimes struggle with the difference.
If there's absence of evidence, you need more evidence!
Good letter.
Big difference between "absence of evidence" and "evidence of absence", and even academics sometimes struggle with the difference.
If there's absence of evidence, you need more evidence!
Good letter.
I went back in time as far as I could go, and the first post was DELETED.
bsky.app/profile/mlob...
Damn.
bsky.app/profile/mlob...
Damn.
December 15, 2025 at 8:40 PM
I went back in time as far as I could go, and the first post was DELETED.
bsky.app/profile/mlob...
Damn.
bsky.app/profile/mlob...
Damn.
I despise that "novelty" is a driver of anything to do with publication in epidemiology.
It is an incentive to p-hack, data mine, fabricate data, conduct analyses seeking specific results etc.
If it is robust and adds to the evidence base, then it should be published.
It is an incentive to p-hack, data mine, fabricate data, conduct analyses seeking specific results etc.
If it is robust and adds to the evidence base, then it should be published.
Analyses of open health datasets can offer valuable contributions to health & medicine, but recent years have seen a proliferation of papers lacking robust or novel findings. In this Editorial, we provide guidance for conducting & reporting high-quality analyses using open datasets🧪
plos.io/4poBmZo
plos.io/4poBmZo
Setting the standard for high-quality studies using open health datasets
Large open health datasets present unique opportunities for studies that when well-designed, conducted, and reported, can offer valuable contributions to health and medicine. However, recent years hav...
plos.io
December 15, 2025 at 3:08 PM
I despise that "novelty" is a driver of anything to do with publication in epidemiology.
It is an incentive to p-hack, data mine, fabricate data, conduct analyses seeking specific results etc.
If it is robust and adds to the evidence base, then it should be published.
It is an incentive to p-hack, data mine, fabricate data, conduct analyses seeking specific results etc.
If it is robust and adds to the evidence base, then it should be published.
Reposted by Sean Harrison, PhD
King Charles argued on Friday that, thanks to early diagnosis, his treatment will be reduced in the New Year.
He urged people to take up cancer screening saying: “Early diagnosis quite simply saves lives.”
www.bbc.co.uk/news/a...
1/10
He urged people to take up cancer screening saying: “Early diagnosis quite simply saves lives.”
www.bbc.co.uk/news/a...
1/10
King Charles 'deeply touched' by reaction to cancer TV message, says Buckingham Palace
In a TV broadcast on Friday night, the King said an early diagnosis was key to the "good news" that his treatment is being scaled back.
www.bbc.co.uk
December 14, 2025 at 8:02 AM
King Charles argued on Friday that, thanks to early diagnosis, his treatment will be reduced in the New Year.
He urged people to take up cancer screening saying: “Early diagnosis quite simply saves lives.”
www.bbc.co.uk/news/a...
1/10
He urged people to take up cancer screening saying: “Early diagnosis quite simply saves lives.”
www.bbc.co.uk/news/a...
1/10
Reposted by Sean Harrison, PhD
So...my undergrad thesis student is doing a quality analysis of studies found in meta-analyses. She identified a few and we contacted the authors to request their effect sizes and other variables for the studies in their papers.
Here's what happened:
scientiapsychiatrica.com/index.php/Sc...
Here's what happened:
scientiapsychiatrica.com/index.php/Sc...
The Impact of Social Media on Adolescent Mental Health: A Meta-Analysis | Scientia Psychiatrica
Introduction: The proliferation of social media has raised significant concerns about its potential effects on the mental health of adolescents. This meta-analysis aims to provide a comprehensive asse...
scientiapsychiatrica.com
December 7, 2025 at 11:11 PM
So...my undergrad thesis student is doing a quality analysis of studies found in meta-analyses. She identified a few and we contacted the authors to request their effect sizes and other variables for the studies in their papers.
Here's what happened:
scientiapsychiatrica.com/index.php/Sc...
Here's what happened:
scientiapsychiatrica.com/index.php/Sc...
Going through references is such a mind-numbing, irritating (paywalls, shitty journals, etc.), and thankless task, but absolutely necessary to ensure statements are backed up by evidence.
I wrote about this a while ago when trying to find evidence for a claim:
seanharrison.blog/2025/02/21/i...
I wrote about this a while ago when trying to find evidence for a claim:
seanharrison.blog/2025/02/21/i...
December 1, 2025 at 9:23 AM
Going through references is such a mind-numbing, irritating (paywalls, shitty journals, etc.), and thankless task, but absolutely necessary to ensure statements are backed up by evidence.
I wrote about this a while ago when trying to find evidence for a claim:
seanharrison.blog/2025/02/21/i...
I wrote about this a while ago when trying to find evidence for a claim:
seanharrison.blog/2025/02/21/i...
All screening causes harm.
Some screening causes benefit.
You need studies and stats to tell you whether the benefit outweighs the harm.
Some screening causes benefit.
You need studies and stats to tell you whether the benefit outweighs the harm.
November 28, 2025 at 1:55 PM
All screening causes harm.
Some screening causes benefit.
You need studies and stats to tell you whether the benefit outweighs the harm.
Some screening causes benefit.
You need studies and stats to tell you whether the benefit outweighs the harm.
IHT at 100%: 50% to the Government as tax, 50% into a fund to provide every kid with a "starting out" grant at 18 years old.
Tackle inequality at source and help out people just starting, many of whom will need the money most.
"But they'll just set up trusts!" -- k, guess we'll do nothing!
Tackle inequality at source and help out people just starting, many of whom will need the money most.
"But they'll just set up trusts!" -- k, guess we'll do nothing!
one of my most contrarian thoughts is to tax IHT at 100% - & use the money to lower tax on actually working for a living
November 24, 2025 at 3:18 PM
IHT at 100%: 50% to the Government as tax, 50% into a fund to provide every kid with a "starting out" grant at 18 years old.
Tackle inequality at source and help out people just starting, many of whom will need the money most.
"But they'll just set up trusts!" -- k, guess we'll do nothing!
Tackle inequality at source and help out people just starting, many of whom will need the money most.
"But they'll just set up trusts!" -- k, guess we'll do nothing!
I believe, for policy, the effect size doesn't matter by itself (unless it's 0): it only matters in conjunction with costs and QALYs (or similar), including side-effects.
Cost-effectiveness matters most: a smaller effect size but better side-effect profile or cheaper intervention may be better.
Cost-effectiveness matters most: a smaller effect size but better side-effect profile or cheaper intervention may be better.
Most scientists don't understand how effect sizes work and are therefore far too quick to dismiss "small" effects.
A correlation of .03 between taking aspirin & prevention of future heart attacks implied the prevention of 85 attacks in a sample of 10,845 people
journals.sagepub.com/doi/10.1177/...
A correlation of .03 between taking aspirin & prevention of future heart attacks implied the prevention of 85 attacks in a sample of 10,845 people
journals.sagepub.com/doi/10.1177/...
November 24, 2025 at 9:12 AM
I believe, for policy, the effect size doesn't matter by itself (unless it's 0): it only matters in conjunction with costs and QALYs (or similar), including side-effects.
Cost-effectiveness matters most: a smaller effect size but better side-effect profile or cheaper intervention may be better.
Cost-effectiveness matters most: a smaller effect size but better side-effect profile or cheaper intervention may be better.
Reposted by Sean Harrison, PhD
Ok, I've thought about this for too long, and come to yet another conclusion about Lord's paradox.
seanharrison.blog/2025/10/12/y...
seanharrison.blog/2025/10/12/y...
October 12, 2025 at 3:06 PM
Ok, I've thought about this for too long, and come to yet another conclusion about Lord's paradox.
seanharrison.blog/2025/10/12/y...
seanharrison.blog/2025/10/12/y...
Ok, I've thought about this for too long, and come to yet another conclusion about Lord's paradox.
seanharrison.blog/2025/10/12/y...
seanharrison.blog/2025/10/12/y...
October 12, 2025 at 3:06 PM
Ok, I've thought about this for too long, and come to yet another conclusion about Lord's paradox.
seanharrison.blog/2025/10/12/y...
seanharrison.blog/2025/10/12/y...
I hadn't heard of Lord's paradox before, so looked it up.
Pearl gives Lord's version in full.
www.degruyterbrill.com/document/doi...
What struck me was that I didn't see "regression to the mean" in either the wiki article or Pearl's paper, but... the second statistician is ignoring that?
Pearl gives Lord's version in full.
www.degruyterbrill.com/document/doi...
What struck me was that I didn't see "regression to the mean" in either the wiki article or Pearl's paper, but... the second statistician is ignoring that?
October 9, 2025 at 4:01 PM
I hadn't heard of Lord's paradox before, so looked it up.
Pearl gives Lord's version in full.
www.degruyterbrill.com/document/doi...
What struck me was that I didn't see "regression to the mean" in either the wiki article or Pearl's paper, but... the second statistician is ignoring that?
Pearl gives Lord's version in full.
www.degruyterbrill.com/document/doi...
What struck me was that I didn't see "regression to the mean" in either the wiki article or Pearl's paper, but... the second statistician is ignoring that?
I think the key thing in using LLMs to write code is the answer to this question:
"Why have you written this line of code?"
If the answer is: "Because I needed a logistic regression controlling for [X] to give an OR, interpreted as...", great, no different to using StackOverflow...
"Why have you written this line of code?"
If the answer is: "Because I needed a logistic regression controlling for [X] to give an OR, interpreted as...", great, no different to using StackOverflow...
Y'all. I just got ChatGPT to do everything in R for this manuscript. I mean EVERYTHING. And it's all legit and reproducible. I'm shook.
How are we mentoring our trainees in statistics now? Who needs to learn coding in R line by line, and who doesn't?
scienceforeveryone.science/statistics-i...
How are we mentoring our trainees in statistics now? Who needs to learn coding in R line by line, and who doesn't?
scienceforeveryone.science/statistics-i...
Statistics in the era of AI
How do we mentor, teach, and do stats when AI can do so much of the work?
scienceforeveryone.science
October 9, 2025 at 3:10 PM
I think the key thing in using LLMs to write code is the answer to this question:
"Why have you written this line of code?"
If the answer is: "Because I needed a logistic regression controlling for [X] to give an OR, interpreted as...", great, no different to using StackOverflow...
"Why have you written this line of code?"
If the answer is: "Because I needed a logistic regression controlling for [X] to give an OR, interpreted as...", great, no different to using StackOverflow...
Something I've thought about but haven't seen written yet:
There *should* be an observational association between autistic children and pain relief in pregnancy.
Austism is highly genetic, so the chance of mothers being autistic is high (even if undiagnosed).
There *should* be an observational association between autistic children and pain relief in pregnancy.
Austism is highly genetic, so the chance of mothers being autistic is high (even if undiagnosed).
I read close to 50 papers in order to give an evidence-based, accurate response to Monday's press conference.
There's a lot to unpack here. I hope you find this informative.
Cc: @mcgilloss.bsky.social
www.mcgill.ca/oss/article/...
There's a lot to unpack here. I hope you find this informative.
Cc: @mcgilloss.bsky.social
www.mcgill.ca/oss/article/...
All About the Tylenol-Autism Brouhaha
At an unhinged press conference last Monday, Donald Trump and his medical acolytes revealed an actionable cause of autism at long last: Tylenol. In what felt like a social media post come to life, Tru...
www.mcgill.ca
September 29, 2025 at 9:57 AM
Something I've thought about but haven't seen written yet:
There *should* be an observational association between autistic children and pain relief in pregnancy.
Austism is highly genetic, so the chance of mothers being autistic is high (even if undiagnosed).
There *should* be an observational association between autistic children and pain relief in pregnancy.
Austism is highly genetic, so the chance of mothers being autistic is high (even if undiagnosed).
This (the idea of using LLMs as stand-ins for humans) is so very, very dumb.
Legitimately surprised there's enough published work using this method that it needed refuting, but good work for doing so!
Legitimately surprised there's enough published work using this method that it needed refuting, but good work for doing so!
Can large language models stand in for human participants?
Many social scientists seem to think so, and are already using "silicon samples" in research.
One problem: depending on the analytic decisions made, you can basically get these samples to show any effect you want.
THREAD 🧵
Many social scientists seem to think so, and are already using "silicon samples" in research.
One problem: depending on the analytic decisions made, you can basically get these samples to show any effect you want.
THREAD 🧵
The threat of analytic flexibility in using large language models to simulate human data: A call to attention
Social scientists are now using large language models to create "silicon samples" - synthetic datasets intended to stand in for human respondents, aimed at revolutionising human subjects research. How...
arxiv.org
September 18, 2025 at 9:29 AM
This (the idea of using LLMs as stand-ins for humans) is so very, very dumb.
Legitimately surprised there's enough published work using this method that it needed refuting, but good work for doing so!
Legitimately surprised there's enough published work using this method that it needed refuting, but good work for doing so!
Reposted by Sean Harrison, PhD
John Finnemore's Souvenir Programme back for 2025
John Finnemore's Souvenir Programme back for 2025
The best of the week's comedy on TV, radio and streaming
dlvr.it
August 24, 2025 at 10:03 AM
John Finnemore's Souvenir Programme back for 2025
p=1, bitches
New rule: Not engaging with people who can not understand that when you randomize, the pre-experiment population means in each group are the same and that any differences are due to sampling variability.
August 16, 2025 at 6:46 PM
p=1, bitches
Apt.
I didn't know Sean Connery was dead!
I didn't know Sean Connery was dead!
August 15, 2025 at 12:38 PM
Apt.
I didn't know Sean Connery was dead!
I didn't know Sean Connery was dead!
It's the same mentality as:
"Group has X quality, therefore any member of that group has that quality"
(notwithstanding the "group has X quality" is often wrong)
At heart, it's laziness, same as every prejudicial -ism.
Even if the first statement is correct, the second doesn't follow.
"Group has X quality, therefore any member of that group has that quality"
(notwithstanding the "group has X quality" is often wrong)
At heart, it's laziness, same as every prejudicial -ism.
Even if the first statement is correct, the second doesn't follow.
this is excellent. some junior colleagues keep thinking high IF = good paper, and im trying to explain repeatedly that this is not a good way to appraise a publication
This is an excellent illustration of how useless the "Impact Factor" metric is.
The most influential multidisciplinary chem journals are JACS and Angewandte. They did not make the list.
In this short ad hoc 🧵 I will analyze in real time what made the list and why. I have a bad feeling about this.
The most influential multidisciplinary chem journals are JACS and Angewandte. They did not make the list.
In this short ad hoc 🧵 I will analyze in real time what made the list and why. I have a bad feeling about this.
August 14, 2025 at 11:03 AM
It's the same mentality as:
"Group has X quality, therefore any member of that group has that quality"
(notwithstanding the "group has X quality" is often wrong)
At heart, it's laziness, same as every prejudicial -ism.
Even if the first statement is correct, the second doesn't follow.
"Group has X quality, therefore any member of that group has that quality"
(notwithstanding the "group has X quality" is often wrong)
At heart, it's laziness, same as every prejudicial -ism.
Even if the first statement is correct, the second doesn't follow.
Reposted by Sean Harrison, PhD
A letter from myself and @jdwilko.bsky.social that expands on this a bit more. Thanks to Fertility and Sterility for the opportunity.
doi.org/10.1016/j.fe...
doi.org/10.1016/j.fe...
August 12, 2025 at 9:54 AM
A letter from myself and @jdwilko.bsky.social that expands on this a bit more. Thanks to Fertility and Sterility for the opportunity.
doi.org/10.1016/j.fe...
doi.org/10.1016/j.fe...