Test severity may at once improve the ability of a replication study to reduce uncertainty, but it can also leave less uncertainty to be reduced to begin with.

My paper with @lakens.bsky.social and @annaveer.bsky.social - “Replication value as a function of citation impact and sample size” - has just been published in Meta psychology! open.lnu.se/index.php/me...

Extremely honored to recieve Oslo New University College's science award for 2025. ONH has been a fantastic base to conduct my research at for the past 4 years, and I have an amazing team of colleagues around me to thank for that. From the bottom of my heart, thank you all!

Later on the authors recommend abductive inference. I agree with the recommendation. However, abduction that leads us to a causal conclusion is just causal inference by another name. Saying that causal inference is not allowed is not sage advice. Better to emphasize that causal inference is hard.

Can we infer causal relations from undirected network models of mental disorders? Some authors apparently say no. Personally, I don't see what network models are good for if they can't help us understand disease etiology (i.e. causality).

Quote from onlinelibrary.wiley.com/doi/10.1002/...

New blog post! Why experiments are the gold standard for answering causal questions (pedermisager.org/blog/why-exp...). Many text books insist on experimental evidence to draw causal inferences bvut don't fully explain exactly what gives experiments their special powers.

From www.chronicle.com/article/soci... by Gelman and King. I applaud the idea presented in this piece, but this specific argument is flawed🧵

TST overcomes this limitation. TST has all the benefits of TOST, plus the ability to formally detect precisely measured large effects, with no loss in statistical power compared to TOST. TST is a simple upgrade of TOST, and can replace TOST+NHST whenever a SESOI can be specified.

Two-one-sided equivalence testing (TOST) solves this problem, but can in turn never show that an effect is larger than SESOI. TOST cannot tell the difference between an accurate estimate of a large effect (blue) and an inaccurate estimate of a potentially small effect (orange).

A traditional null-hypothesis test (NHST) can never accept H0, because it cannot tell the difference between an accurate estimate of a small/null effect (blue) and an inaccurate estimate of a potentially large effect (orange).

A three-sided test (TST) combines tests for equivalence, superiority and inferiority to let us decide if the observed effect is smaller or larger than the smallest effect size of interest. Researchers often use TOST+NHST for such decision, but this is a mistake.

3/
We also wrote a Transparency statement for this output, and added roles to #CredITTaxonomy for hackathon participation (credit.niso.org). If you have thoughts about this statement or ideas about other things we could have mentioned, let us know!

Ever had really great discussions during a hackathon and gone home thinking others could also benefit from the ideas bounced around? Inspired by @simine.com's garage analogy, we thought about how to coordinate scientific self-correction. See our ‘hackathon proceedings’ osf.io/preprints/me...
1/

Interested in replication research? Consider writing a commentary for the next special issue in Meta-Psychology which will publish critiques of the article "Replication value as a function of citation impact and sample size": open.lnu.se/index.php/me...

@annaveer.bsky.social @lakens.bsky.social Thed van Leeuwen and I applied a method for quantifying "replication value" on a large sample of fMRI articles in social neuroscience, to see if the method is practically feasible. Open access version of our attempt is now online: doi.org/10.1016/j.co...

The jist of the post is that correlation, by itself, can mean many things. Causation implies correlation, but so does confounding and selection bias. In observational studies, we need to consider at least 5 possible explanations for why any two variables are correlated.