James Steele
@jamessteeleii.bsky.social
700 followers 480 following 4.9K posts
I'm just a guy who's an academic for fun
Posts Media Videos Starter Packs
Pinned
jamessteeleii.bsky.social
Excited to have now started new role as Head of Research with Macrofactor (macrofactorapp.com). Looking forward to getting stuck into some really cool projects and getting that out to all the users!
Macrofactor Logo - An M and F in vertical orientation.
jamessteeleii.bsky.social
I did see recently though a paper that added when discussing it that "it's not been peer reviewed". In fact it's received more review and revision than any other paper I've written 😂
jamessteeleii.bsky.social
One of my most highly cited works is still only a "preprint" and yet it's such that folks working in the field feel they can't not cite and discuss it...
Reposted by James Steele
matti.vuorre.com
Against Publishing: universonline.nl/nieuws/2025/...

Preprints are read, shared, and cited, yet still dismissed as incomplete until blessed by a publisher. I argue that the true measure of scholarship lies in open exchange, not in the industry’s gatekeeping of what counts as published.
jamessteeleii.bsky.social
{renv} needs a more obvious R version warning... more than once I have fucked up a project because I pre-registered the pipelines and everything a while back, then when I finally come back to it to actually finish up the project I completely forget I have updated R and end up somehow with problems.
jamessteeleii.bsky.social
Check out this cute ass copy of On Bullshit I found.
A small red covered anniversary edition book copy of On Bullshit by Harry G Frankfurt
jamessteeleii.bsky.social
Wait long enough and Lidl will have t-tests on offer in the middle aisle.
jamessteeleii.bsky.social
Huh, that's weird... assuming that's a paired t-test and some standard software I'd have assumed it might automatically spit out dz back then, but the conversion from t and df doesn't match. Maybe they already used d_rm or used average variance d? Weird they can't tell you too.
jamessteeleii.bsky.social
Oh don't worry, I'm just as snide and sarcastic on there as anywhere else... most of my posts are of the "why everything is so shit" kind of content with a side of pedagogy 😜
jamessteeleii.bsky.social
I'm kinda glad of the serendipity of LinkedIn becoming the new place at roughly the same time as I shifted to industry. I definitely get more interaction there than here. But, I still hang around here for certain people.
jamessteeleii.bsky.social
p.s. Joslyn and I are now formalising his systematic review, pre-registering, updating anything missed, and will be including his unpublished study alongside existing literature.

10/10
jamessteeleii.bsky.social
Researchers should heed Santayana’s adage: “Those who cannot remember the past are condemned to repeat it.” Before launching a new study, ask — does it have the potential to actually change what we know?

9/10
jamessteeleii.bsky.social
Many assume that any new study “adds” to the cumulative evidence. Technically true — but in practice, it may add almost nothing. When studies are too small to improve precision or meaningfully update belief, they become a poor return on research investment.

8/10
jamessteeleii.bsky.social
That’s not a criticism of Joslyn’s work — it was a solid undergrad project and not in any way wasteful as it served a pedagogical purpose. But if this were a new study by established researchers, you’d have to ask: why bother?

6/10
jamessteeleii.bsky.social
Meanwhile, Joslyn collected new data from 8 participants in a crossover design. We analysed his data using the priors from the meta-analysis. The result? His study barely shifted the posterior beliefs — the prior and posterior distributions almost perfectly overlapped.

5/10
A two-panel forest-style plot showing results from an empirical study comparing the effects of ammonia inhalants, placebo, and control conditions on Peak Force (left) and Rate of Force Development (right). The x-axis represents Standardised Mean Difference (SMD), with dotted vertical lines marking the null region (–0.1 to 0.1). Each panel displays posterior distributions (gray density shapes) with mean estimates, 95% quantile intervals, and 2×log(BF) values indicating Bayes factors relative to the null region.

Peak Force panel:

Placebo vs Control: SMD = –0.08 [–0.28, 0.11], 2×log(BF) = 0.26

Ammonia vs Placebo: SMD = 0.22 [–0.01, 0.47], 2×log(BF) = 1.06

Ammonia vs Control: SMD = 0.14 [0.01, 0.30], 2×log(BF) = 0.91

Rate of Force Development panel:

Placebo vs Control: SMD = –0.08 [–0.28, 0.10], 2×log(BF) = –0.33

Ammonia vs Placebo: SMD = 0.30 [0.09, 0.52], 2×log(BF) = –0.53

Ammonia vs Control: SMD = 0.22 [0.09, 0.35], 2×log(BF) = 0.02

A note below the panels explains that 2×log(BF) values indicate the strength of evidence against the null, following the Kass and Raftery (1995) interpretation scale:

Negative = evidence for the null,

0–2 = weak,

2–6 = positive,

6–10 = strong,

10 = very strong evidence for an effect.

Overall, the posterior distributions overlap substantially, suggesting minimal change in beliefs from the prior evidence.
jamessteeleii.bsky.social
While most individual studies were mostly “null,” collectively they suggested small, possibly positive effects on some outcomes — though with modest precision.

4/10
A three-panel forest-style plot showing results from a contrast-based network meta-analysis comparing the effects of ammonia inhalants, placebo, and control conditions on power, speed, and strength outcomes. Each panel (Power, Speed, Strength) shows standardized mean differences (SMDs) and 95% quantile intervals for each contrast (Placebo vs Control, Ammonia vs Placebo, Ammonia vs Control).

Power:

Placebo vs Control: SMD = –0.09 [–0.26, 0.09], Probability of null effect = 51%, meaningful effect = 2%.

Ammonia vs Placebo: SMD = 0.33 [0.19, 0.46], Probability of null = 0%, meaningful = 100%.

Ammonia vs Control: SMD = 0.23 [0.10, 0.38], Probability of null = 2%, meaningful = 98%.

Speed:

Placebo vs Control: SMD = 0.05 [–0.34, 0.45], Probability of null = 38%, meaningful = 41%.

Ammonia vs Placebo: SMD = 0.02 [–0.35, 0.40], Probability of null = 40%, meaningful = 34%.

Ammonia vs Control: SMD = 0.08 [–0.25, 0.40], Probability of null = 42%, meaningful = 44%.

Strength:

Placebo vs Control: SMD = –0.06 [–0.27, 0.15], Probability of null = 57%, meaningful = 6%.

Ammonia vs Placebo: SMD = 0.18 [0.01, 0.36], Probability of null = 19%, meaningful = 81%.

Ammonia vs Control: SMD = 0.12 [0.03, 0.28], Probability of null = 42%, meaningful = 57%.

Each panel shows posterior distributions as gray density shapes, centered around the estimated SMDs. Dotted vertical lines indicate the null region (–0.1 to 0.1). Text at the bottom notes that probabilities of null and meaningful effects were calculated as the proportion of posterior mass within or beyond this region, respectively.
jamessteeleii.bsky.social
Joslyn conducted the review (notably, existing reviews hadn’t done any quantitative synthesis), identified 11 studies (n = 10–25), extracted data, and we fit contrast-based network meta-analytic models.

3/10
jamessteeleii.bsky.social
He was an excellent student — realistic about recruitment challenges and well ahead of his peers in getting started — so we decided to first do a systematic review and meta-analysis to generate informative priors for analysing his small empirical dataset using Bayesian inference.

2/10
jamessteeleii.bsky.social
Sometimes we already know enough to make further research on a topic a waste of resources.

Here’s an example (🧵1/10)

Joslyn McLeod approached me about doing his undergraduate dissertation on the effects of ammonia inhalants on performance.
jamessteeleii.bsky.social
I might sound a bit egotistical saying this... but it's a scathing indictment of the field of #sportscience #exercisescience that folks get just as (actually more) excited by the latest shitty little n = 10-30 trial as they do about this kind of work 🙃
Reposted by James Steele
liamsatchell.bsky.social
New paper! - I was wrong! We checked and checked again we were definitely wrong.

And I couldn't be happier that we are able to publish these findings at QJEP - Satchell, Hall & Jones 🧵

osf.io/preprints/ps...
OSF
osf.io
jamessteeleii.bsky.social
After this, if we end up finding precise effect estimates very much in line with theory I'm calling it there... further research would genuinely be a waste of time (brief commentary coming soon on this point of research waste too 😜).
jamessteeleii.bsky.social
We have our next Project Discover currently underway conducting another high-powered, pre-registered test of theoretically derived precise predictions regarding comparative effects of training volume:

osf.io/7vmxk/
OSF
osf.io
jamessteeleii.bsky.social
@f2harrell.bsky.social just managed to squeeze in a paraphrase and credit to you for this in a commentary piece!