The Unjournal (Unjournal.org)
banner
unjournal.bsky.social
The Unjournal (Unjournal.org)
@unjournal.bsky.social
Researchers, practitioners, & open science advocates building a better system for research evaluation. Nonprofit. We commission public evaluation & rating of hosted work. To make rigorous research more impactful, & impactful research more rigorous.
Reposted by The Unjournal (Unjournal.org)
FWIW at Unjournal.org we

1. Prioritize research in terms of its potential for impact (see coda.io/d/Public-Dat...)

2. Commission public, expert, evaluations of this research, asking for reports & ratings across several criteria, including Relevance/Usefulness
December 9, 2025 at 4:03 PM
Evaluation Summary and Metrics: "Adjusting for Scale-Use Heterogeneity in Self-Reported Well-Being" (interim eval.)
Evaluation Summary and Metrics: "Adjusting for Scale-Use Heterogeneity in Self-Reported Well-Being" (interim eval.)
Evaluation Summary and Metrics: "Adjusting for Scale-Use Heterogeneity in Self-Reported Well-Being" for The Unjournal.
bit.ly
December 9, 2025 at 3:26 PM
Researchers, research-users, practitioners, funders, managers:
Unjournal.org wants your feedback on our work & how we can improve & boost our impact.

Survey with lottery incentive: See
Stakeholder Feedback: help Unjournal measure & enhance its impact — The Unjournal
We've evaluated 55+ papers/projects across global health, animal welfare, AI governance, and development economics; and more in the pipeline . We've built tools, databases, and partnerships. We’re now focusing on understanding and enhancing our impact — “ is this actually useful, who is using it,
bit.ly
December 4, 2025 at 1:58 PM
Reposted by The Unjournal (Unjournal.org)
This year’s @sjdm-tweets.bsky.social presidential address from @donandrewmoore.bsky.social built on last year’s address from @joesimmons.bsky.social about improving #cogSci.


Don celebrated our normalization of pre-registration and #openAccess data, materials, etc. 🙌

But he reminded us of how...
@joesimmons.bsky.social's 2024 Prez Address at #SJDM was WILD — a behind the scenes of @datacolada.bsky.social:

Takeaways:
- P-curves aren't sufficient
- Share EVERYTHING (including #Qualtrics files)
- Describe findings AS OPERATIONALIZED
- Loads of #JDM is good! (cf. headlines)
- Keep speaking up!
November 23, 2025 at 7:41 PM
Reposted by The Unjournal (Unjournal.org)
...many leaders in the field are still seeking prestige from a few for-profit, rent-seeking journals.



He then pointed us to better publication practices:
- @peercommunityin.bsky.social

- @unjournal.bsky.social

- @metaror.bsky.social

Post credits: @alexh.bsky.social, @jwastrachan.bsky.social
November 23, 2025 at 7:41 PM
How is The Unjournal relevant for #effectivealtruism, what are our plans, and how would we use 'marginal funding'?

See
The Unjournal: Bridging the Rigor/Impact Gaps for EA-relevant Research Questions — EA Forum
Overview The Unjournal is a nonprofit organization (est. 2023) that commissions rigorous public expert evaluations of impactful research. We've built…
bit.ly
November 23, 2025 at 6:48 PM
Reposted by The Unjournal (Unjournal.org)
We *are* doing this.

There is the @unjournal.bsky.social. There is Peer Community In @peercommunityin.bsky.social. There are dozens of newly-founded diamond OA (free to read, free to publish) journals being born, eg Experimental Philosophy @xphilosopher.bsky.social.
November 14, 2025 at 11:20 AM
Unjournal.org is planning to work with:
https://bit.ly/49lWRVJ (Consider pledging?)

Plans:
- Integrate some Unjournal’s Pivotal Question content (cultured meat, plant-based alternatives)
- Help operationalize & vet tournament questions
Metaculus' First Animal-Focused Forecasting Tournament
Project summary This project aims to run the first animal-focused forecasting tournament on the Metaculus platform. The goal is to generate forecasts on decision-relevant questions that will help both funders and animal-aligned organisations make more informed decisions about their funding allocation and strategy. Questions will be crowdsourced from the collective animal advocacy community and vetted appropriately to ensure the questions’ relevancy to decision-making in the community. The tournament contains a prize pool, which will be split amongst the best forecasters. What are this project's goals? How will you achieve them? The primary goals of this project are threefold: To enable funders in the animal welfare/rights space to prioritize animal-aligned interventions more effectively. To raise awareness amongst animal-aligned organisations about forecasting, to enable them to make better-informed decisions about their strategies and future direction. We also want forecasters outside the animal advocacy community to weigh in on the forecasts, diversifying the set of forecasters for the questions in the tournament. #1 and #2 will primarily be achieved by advertising the tournament in various animal advocacy communities (e.g. Hive, Sentient Futures) and partnering with a media body like Sentient Media. I’m hoping to achieve #3 by the presence of a prize pool in the tournament, so if forecasters weren’t intrinsically motivated to forecast these questions, at least the money to be won would be a good motivator. How will this funding be used? The funding will be used to create a prize pool for the Metaculus tournament. If funding exceeds $10,000, a small portion of the amount above the $10,000 will also be used to cover my efforts in filtering questions, operationalising them to a Metaculus-friendly format, advertising the tournament, and following up with decision makers in the animal advocacy community to gauge whether they have found the forecasts useful. The minimum funding is set such that the prize pool will hopefully be viable to incentivize a sufficient number of forecasters to predict. More money raised will produce a larger prize pool, greater participation incentive, and more attention on the tournament, and will also enable more outreach to spread awareness. Who is on your team? What's your track record on similar projects? It’s primarily me on the team. Jeffrey Maxim, a forecasting enthusiast and blogger from the Hive community, and Ryan Beck, the Forecasting Programs Coordinator at Metaculus are playing supporting roles. The team at The Unjournal will also be collaborating with me on selecting and refining the final set of questions. I have not personally worked on something similar, but Ryan Beck is experienced in shaping and operationalising Metaculus tournaments, including many of Metaculus’ active and past tournaments. What are the most likely causes and outcomes if this project fails? Questions evolve to become non-decision-relevant over time. Funders and animal organisations are unsure how to translate forecasts into concrete actions/decisions Aggregate forecasts on the tournament questions are completely unreflective of the actual outcome, leading decision makers to potentially make the wrong decisions How much money have you raised in the last 12 months, and from where? None.
bit.ly
November 15, 2025 at 10:40 PM
We are delighted to welcome Paolo Crosetto to our advisory board at The Unjournal. Paolo is a full-time senior researcher (Directeur de Recherche, DR) at INRAE. He works in Grenoble, France, within the Research Unit GAEL – Grenoble Applied Economics Laboratory.
November 13, 2025 at 5:03 PM
How long do Unjournal evaluators take to do their evaluations? For our academic stream, about 8 hours on average, with substantial dispersion.

(Scatterplot: imputed data from 40 quantifiable responses, not including applied stream nor those who used the PubPub form.)
November 12, 2025 at 11:48 PM
Looking for examples of high-quality peer reviews of impactful research in economics & social science? See Unjournal's "commended evaluations": https://bit.ly/4qJo0rX

Recently commended work by 3 David's (Reiley, Hugh-Jones, & Manheim), Estere Seinkmane, Eleanor Tsai, & 2 anonymous heroes.
Overview · Prize-winning & commended evaluations
bit.ly
November 7, 2025 at 3:08 PM
Researchers: Has your paper been evaluated by The Unjournal (see unjournal.pubpub.org)?

We want your feedback on the experience, the quality of the evaluations, and how this impacted your work.

https://bit.ly/4qKtYIZ

1/10 get a $100 prize.
Author Survey Form
Please note that all questions are optional. By completing this form, you will enter a draw with a chance of winning $100. Every 1 out of 10 respondents will receive the compensation.
bit.ly
November 6, 2025 at 4:36 PM
We're inviting academic journals to recognize The Unjournal's public evaluation packages in their editorial process.

- Help w/ editorial triage, review quality, & turnaround
- Support transparency, open-access.

Details: https://bit.ly/3JEee9J

Feedback & questions welcome!
Traditional journals can use Unjournal evaluations | The Unjournal: project and communication space
bit.ly
November 5, 2025 at 10:48 PM
All of this is a small fraction of the billions paid to for-profit publishers for "Article Processing Charges", of course.

opusproject.eu/openscience-...

(Maybe $50 million per year in APCs for physics and astronomy the most active areas for SciPost)
October 30, 2025 at 8:15 PM
If you're among the 2000+ authors citing research evaluated by Unjournal.org, go to unjournal.pubpub.org to learn how commissioned experts rated & assessed that research.

Top 'citers of unjournal-evaluated research': https://bit.ly/3WRHc8W include Esther Duflo, Julian Jamison, & Berk Özler
October 22, 2025 at 4:16 PM
Congrats to Mokyr, Aghion & Howitt for the Nobel memorial prizes in economics sciences!

And, you guessed it: This is a blatant promo for one of our first & strongest evaluation packages, of "Artificial Intelligence & Economic Growth” by Aghion et al.
“Artificial Intelligence and Economic Growth”: Evaluation summary & metrics
Summary, metrics and ratings, and Manager's comments on Evaluation of “Artificial Intelligence and Economic Growth” by Aghion et al.
bit.ly
October 13, 2025 at 4:49 PM
Evaluation Summary and Metrics: "Does online fundraising increase charitable giving? A nationwide field experiment on Facebook"
Evaluation Summary and Metrics: "Does online fundraising increase charitable giving? A nationwide field experiment on Facebook"
Evaluation Summary and Metrics: ""Does online fundraising increase charitable giving? A nationwide field experiment on Facebook"" for The Unjournal.
bit.ly
October 10, 2025 at 7:44 PM
Reposted by The Unjournal (Unjournal.org)
Does "Irrigation strengthen climate resilience"? -- Unjournal evaluation

https://bit.ly/3J1iMqk

BenYishay et al '24: River-based irrigation boosted #agricultural output & child #nutrition & reduced conflict in supported communities, but had negative spillovers on surrounding areas.
--->
October 8, 2025 at 9:34 PM
Does "Irrigation strengthen climate resilience"? -- Unjournal evaluation

https://bit.ly/3J1iMqk

BenYishay et al '24: River-based irrigation boosted #agricultural output & child #nutrition & reduced conflict in supported communities, but had negative spillovers on surrounding areas.
--->
October 8, 2025 at 9:34 PM
Evaluation Summary and Metrics: "Irrigation Strengthens Climate Resilience: Long-term Evidence from Mali using Satellites and Surveys"
Evaluation Summary and Metrics: "Irrigation Strengthens Climate Resilience: Long-term Evidence from Mali using Satellites and Surveys"
Evaluation Summary and Metrics: "Irrigation Strengthens Climate Resilience: Long-term Evidence from Mali using Satellites and Surveys" for The Unjournal.
bit.ly
October 8, 2025 at 8:40 PM
I love what #SciPost does & their narrative.
Much agrees w/ Unjournal.org practices.

Some differences (Us/Them)
1. Quant. ratings vs accept/reject
2. Publish all evaluations vs. only if ‘accepted’
3. Fields & applied focus
4. We pay evaluators
5. We don’t 'publish' papers, just evaluations
October 3, 2025 at 6:25 PM