Against Publishing: universonline.nl/nieuws/2025/...
Preprints are read, shared, and cited, yet still dismissed as incomplete until blessed by a publisher. I argue that the true measure of scholarship lies in open exchange, not in the industry’s gatekeeping of what counts as published.
Preprints are read, shared, and cited, yet still dismissed as incomplete until blessed by a publisher. I argue that the true measure of scholarship lies in open exchange, not in the industry’s gatekeeping of what counts as published.
Very useful extension! I used an earlier version in this paper: pgmj.github.io/rasch_itemfit/
Do you have one (or more) key references for that? Would be nice to have handy as ppl keep being curious about these models
Re-updated the page and the R package, including the conditional reliability method by McNeish & Dumas as well.
McNeish & Dumas (2025). Reliability representativeness: How well does coefficient alpha summarize reliability across the score distribution? doi.org/10.3758/s134...
McNeish & Dumas (2025). Reliability representativeness: How well does coefficient alpha summarize reliability across the score distribution? doi.org/10.3758/s134...
Reliability representativeness: How well does coefficient alpha summarize reliability across the score distribution? - Behavior Research Methods
Scale scores in psychology studies are commonly accompanied by a reliability coefficient like alpha. Coefficient alpha is an index that summarizes reliability across the entire score distribution, implying equal precision for all scores. However, an underappreciated fact is that reliability can be conditional such that scores in certain parts of the score distribution may be more reliable than others. This conditional perspective of reliability is common in item response theory (IRT), but psychologists are generally not well versed in IRT. Correspondingly, the representativeness of a single summary index like alpha across the entire score distribution can be unclear but is rarely considered. If conditional reliability is fairly homogeneous across the score distribution, coefficient alpha may be sufficiently representative and a useful summary. But, if conditional reliability is heterogeneous across the score distribution, alpha may be unrepresentative and may not align with the reliability of a typical score in the data or with a particularly important score like a cut point where decisions are made. This paper proposes a method, R package, and Shiny application to quantify the potential differences between coefficient alpha and conditional reliability across the score distribution. The goal is to facilitate comparisons between conditional reliability and reliability summary indices so that psychologists can contextualize the reliability of their scores more clearly and comprehensively.
doi.org
I agree. #openscience combined with #opensource software is great!
The RelRep.R function "just works" with a dataframe with items as columns and produces pretty neat output for alpha or omega. Point estimate and CI for alpha is almost identical to RMU for my example with `eRm::raschdat1[,1:20]` data. github.com/melissagwolf...
Reposted by Magnus Johansson
This is what I find awesome (in the truest sense) about scientific development in the modern age. One group posts a new technique and within a few days it is implemented by others. #rstats is magic (and efficient)
Looking closer at McNeish & Dumas (2025). How well does coefficient alpha summarize reliability across the score distribution? doi.org/10.3758/s134...
The RelRep.R function seems like it could be combined with your approach as a conditional reliability metric. I'll probably have a go at it.
The RelRep.R function seems like it could be combined with your approach as a conditional reliability metric. I'll probably have a go at it.
Reliability representativeness: How well does coefficient alpha summarize reliability across the score distribution? - Behavior Research Methods
Scale scores in psychology studies are commonly accompanied by a reliability coefficient like alpha. Coefficient alpha is an index that summarizes reliability across the entire score distribution, implying equal precision for all scores. However, an underappreciated fact is that reliability can be conditional such that scores in certain parts of the score distribution may be more reliable than others. This conditional perspective of reliability is common in item response theory (IRT), but psychologists are generally not well versed in IRT. Correspondingly, the representativeness of a single summary index like alpha across the entire score distribution can be unclear but is rarely considered. If conditional reliability is fairly homogeneous across the score distribution, coefficient alpha may be sufficiently representative and a useful summary. But, if conditional reliability is heterogeneous across the score distribution, alpha may be unrepresentative and may not align with the reliability of a typical score in the data or with a particularly important score like a cut point where decisions are made. This paper proposes a method, R package, and Shiny application to quantify the potential differences between coefficient alpha and conditional reliability across the score distribution. The goal is to facilitate comparisons between conditional reliability and reliability summary indices so that psychologists can contextualize the reliability of their scores more clearly and comprehensively.
doi.org
I've updated the reliability section of my `easyRasch` vignette as well: pgmj.github.io/raschrvignet...
#psychometrics #rasch #rstats
#psychometrics #rasch #rstats
easyRasch vignette – R, Rasch, etc
R package for Rasch analysis
pgmj.github.io
I don't think so. This is a great development you've made! And through your references on Test Information Function I abandoned that path. As additional reliability metrics, I would like to find 1) a conditional reliability metric (non-constant); 2) a test reliability metric (sample independent)
This is really neat. I have borrowed the reliability() function to my `easyRasch` package, and use plausible values instead of fully Bayesian estimation to produce similar estimates/CIs, see code example below. RMU point estimates are similar to EAP reliability.
pgmj.github.io/easyRasch/re...
pgmj.github.io/easyRasch/re...
CSV will not retain factor levels/labels, etc. I prefer to use .parquet or .qs2 formats, both open source and available in R packages `arrow` and `qs2`. The latter is R specific and has the added benefit of being able to save any R object to a compressed file.
Det är mest troligt s.k. attention-control-frågor, som ibland läggs in i (långa) frågebatterier för att identifiera respondenter som bara kryssar random svar.
Reposted by Magnus Johansson, Jon Green, Dan MacLean
I really really really liked The Oatmeal’s take on AI here theoatmeal.com/comics/ai_art
A cartoonist's review of AI art - The Oatmeal
This is a comic about AI art.
theoatmeal.com
Reposted by Magnus Johansson
”AI isn't going to wake up, become superintelligent and turn you into paperclips – but rich people with AI investor psychosis are almost certainly going to make you much, much poorer.”
pluralistic.net/2025/09/27/e...
pluralistic.net/2025/09/27/e...
Pluralistic: The real (economic) AI apocalypse is nigh (27 Sep 2025) – Pluralistic: Daily links from Cory Doctorow
pluralistic.net
Reposted by Magnus Johansson
#statstab #432 PsychOpen Gold
Thoughts: instead of submitting to greedy and unhelpful publishers, try this list of fully open and free journals in psychology.
#OpenScience #openaccess #apcs #goldaccess #pedagogy
psychopen.eu
Thoughts: instead of submitting to greedy and unhelpful publishers, try this list of fully open and free journals in psychology.
#OpenScience #openaccess #apcs #goldaccess #pedagogy
psychopen.eu
PsychOpen GOLD: Open Access Publishing
We are a Diamond Open Access platform for psychology research. Peer-reviewed, free-to-read journals with no publication fees, promoting open science.
psychopen.eu
Reposted by Magnus Johansson
Love this paper by Hinne doi.org/10.1177/2515... but curious why BMA isn't compatible with competing theories: "BMA is less useful when...each candidate model may represent a different theory of a physical process...the models are not a nuisance factor; they are the focus of the analysis." #statsky
A Conceptual Introduction to Bayesian Model Averaging - Max Hinne, Quentin F. Gronau, Don van den Bergh, Eric-Jan Wagenmakers, 2020
Many statistical scenarios initially involve several candidate models that describe the data-generating process. Analysis often proceeds by first selecting the ...
doi.org
My article "Data is not available upon request" was published in Meta-Psychology. Very happy to see this out!
open.lnu.se/index.php/me...
open.lnu.se/index.php/me...
LnuOpen
| Meta-Psychology
open.lnu.se
Reposted by Magnus Johansson
Exceptional Doctorow piece on the solar rollout, why it's unstoppable, & why many old downsides of solar aren't factors anymore. The entire material mining needed 4 a global solar transition = only 17% of the fossil fuel mining we do every year! & then we're done! doctorow.medium.com/https-plural...
Decarbonization at a distance
A post-American century that runs on sunshine.
doctorow.medium.com
I'm sure there are ways it is brilliant, but my first methods check in EFA/CFA psychometric papers is always whether rule-of-thumb critical values are used to assess model fit. I don't think we should trust model fit interpretations using rule-of-thumb cutoffs in any paper.
Reposted by Magnus Johansson, Matti Vuorre
New preprint with @rogierk.bsky.social @paulbuerkner.com - we introduce "relative measurement uncertainty" - a reliability estimation method that's applicable across a broad class of Bayesian measurement models (e.g., generative-, computational- and item response theory-models osf.io/h54k8
OSF
osf.io
I strongly recommend to stop using rule of thumb cutoffs such as Hu and Bentler since they are unlikely to be appropriate. Dynamic/adaptive cutoffs should be used, see for instance pubmed.ncbi.nlm.nih.gov/36787513/
dynamic : An R Package for Deriving Dynamic Fit Index Cutoffs for Factor Analysis - PubMed
To evaluate the fit of a confirmatory factor analysis model, researchers often rely on fit indices such as SRMR, RMSEA, and CFI. These indices are frequently compared to benchmark values of .08, .06, ...
pubmed.ncbi.nlm.nih.gov
by Matti Vuorre — Reposted by Magnus Johansson
100% this, especially with Wiley given their anti-preprint stance.
Interesting paper on how software could help analysts. ”Causal clarity in statistical software” (Korf et al., 2025). I like many of the ideas, such as clarifying estimand, using multiple estimators, and no output of coefficients for adjustment variables doi.org/10.1093/ije/... #stats #rstats
Causal clarity in statistical software
Imagine running a simple regression in any statistical software of choice—but this time, you only get a point estimate of the regression coefficient. There
doi.org
I'm exited to announce a new resource about making slides with quarto and revealjs. This book is the combination of all the work I have done in this area, reordered and polished up
There isn't a lot of new information yet, but this format allows me to add more easily
slidecrafting-book.com
#quarto
There isn't a lot of new information yet, but this format allows me to add more easily
slidecrafting-book.com
#quarto
Reposted by Magnus Johansson, Duncan Bell, Mary Burke , and 1 more Magnus Johansson, Duncan Bell, Mary Burke, Victoria Moul
I hope Melvin Bragg's retirement will not end In Our Times. He has a lovely voice and manner but it's the people he invites that hold the show. I'll take this chance to post a thread of some of the more memorable episodes for me. Starting with Schaffer, Worrall, and @michelamassimi.bsky.social
BBC Radio 4 - In Our Time, The Scientific Method
Melvyn Bragg and his guests discuss the Scientific Method.
www.bbc.co.uk
Indeed, and since data is always collected within a known time-frame, there is also a known upper bound, which count models generally do not respect. The impact of ignoring the upper bound (and potential one-inflation) will of course depend on the distribution.
Reposted by Magnus Johansson
This is why I wrote a paper on count regression, and in general am a big proponent of generalised linear models. Almost nothing is 'normal'/gaussian. Days off sick are very obviously a count (whole numbers only, never below zero) doi.org/10.1080/2164...
