Lightnews — Scholar-powered news

The Unjournal (Unjournal.org) @unjournal.bsky.social · 1d

Congrats to Mokyr, Aghion & Howitt for the Nobel memorial prizes in economics sciences!

And, you guessed it: This is a blatant promo for one of our first & strongest evaluation packages, of "Artificial Intelligence & Economic Growth” by Aghion et al.

“Artificial Intelligence and Economic Growth”: Evaluation summary & metrics

Summary, metrics and ratings, and Manager's comments on Evaluation of “Artificial Intelligence and Economic Growth” by Aghion et al.

bit.ly

2

The Unjournal (Unjournal.org) @unjournal.bsky.social · 3d

The whole process is a waste of resources and diverts effort from improving research to journal shopping. Just give the paper a public rating, don’t “reject or accept”. That’s what we do at Unjournal.org.

Unjournal | Transparent Evaluation of Impactful Research

The Unjournal enables open, expert evaluation of economics and social science research, without the constraints of traditional journals. Discover how we make rigorous research more transparent.

Unjournal.org

The Unjournal (Unjournal.org) @unjournal.bsky.social · 4d

Evaluation Summary and Metrics: "Does online fundraising increase charitable giving? A nationwide field experiment on Facebook"

Evaluation Summary and Metrics: ""Does online fundraising increase charitable giving? A nationwide field experiment on Facebook"" for The Unjournal.

bit.ly

1

Reposted by The Unjournal (Unjournal.org)

The Unjournal (Unjournal.org) @unjournal.bsky.social · 6d

Does "Irrigation strengthen climate resilience"? -- Unjournal evaluation

https://bit.ly/3J1iMqk

BenYishay et al '24: River-based irrigation boosted #agricultural output & child #nutrition & reduced conflict in supported communities, but had negative spillovers on surrounding areas.
--->

1 1

The Unjournal (Unjournal.org) @unjournal.bsky.social · 6d

#development #ClimateAdaptation #agricultureRetry

This paper was suggested to us by www.aiddata.org/ (by 1 author) as being among their highest-value work meriting further evaluation. (Thanks!)

See our ‘evaluating research from impact-oriented organizations’ initiative -- bit.ly/4oahxEc.

AidData — A Research Lab at William & Mary

We equip policymakers and practitioners with better evidence to improve how sustainable development investments are targeted, monitored, and evaluated. We use rigorous methods, cutting-edge tools, an...

www.aiddata.org

The Unjournal (Unjournal.org) @unjournal.bsky.social · 6d

Eva, managers also flagged identification concerns:

- Potential migration/sorting bias (healthier people moving TO irrigated areas?)

- Program timing may be endogenous (placed where conflict already decreasing?)

- Pre-trends need tighter verification (evidence of boundedly small differences)

1 2

The Unjournal (Unjournal.org) @unjournal.bsky.social · 6d

E1's suggestions:

- Update to modern TWFE methodology

- Quantify output effects "in an area with a larger buffer around the irrigation system"

- Additional robustness checks

1

The Unjournal (Unjournal.org) @unjournal.bsky.social · 6d

Key concerns from E2:

"Need more context-specific information on the causal pathway" for better investment & program design

Measurement issues: surveys used _recall_ on caloric intake rather than standard nutrition indicators

1

The Unjournal (Unjournal.org) @unjournal.bsky.social · 6d

Evaluators liked the:

- Novel combination of spatial + survey data

- Basic empirical patterns (output & nutrition) considered sound

- Important policy question

1

The Unjournal (Unjournal.org) @unjournal.bsky.social · 6d

The evaluations were split:

E1: 75/100 - Strong confidence in agricultural output increases, appreciated the methodology

E2: 46/100 - Found nutrition results "credible" but had substantial doubts about causal interpretation

1

The Unjournal (Unjournal.org) @unjournal.bsky.social · 6d

Does "Irrigation strengthen climate resilience"? -- Unjournal evaluation

https://bit.ly/3J1iMqk

BenYishay et al '24: River-based irrigation boosted #agricultural output & child #nutrition & reduced conflict in supported communities, but had negative spillovers on surrounding areas.
--->

1 1

The Unjournal (Unjournal.org) @unjournal.bsky.social · 6d

Evaluation Summary and Metrics: "Irrigation Strengthens Climate Resilience: Long-term Evidence from Mali using Satellites and Surveys"

Evaluation Summary and Metrics: "Irrigation Strengthens Climate Resilience: Long-term Evidence from Mali using Satellites and Surveys" for The Unjournal.

bit.ly

The Unjournal (Unjournal.org) @unjournal.bsky.social · 11d

@scipost.bsky.social congratulations on your progress & thanks for all you do.

1

The Unjournal (Unjournal.org) @unjournal.bsky.social · 11d

I love what #SciPost does & their narrative.
Much agrees w/ Unjournal.org practices.

Some differences (Us/Them)
1. Quant. ratings vs accept/reject
2. Publish all evaluations vs. only if ‘accepted’
3. Fields & applied focus
4. We pay evaluators
5. We don’t 'publish' papers, just evaluations

1 1 1

The Unjournal (Unjournal.org) @unjournal.bsky.social · 13d

But having a "published object" in the ecosystem linked both to our evaluations and ratings and linked to the author's name could have value in terms of making our evaluations and ratings count career-wise.

The Unjournal (Unjournal.org) @unjournal.bsky.social · 13d

At The Unjournal we don't do the "publish" part of PRC -- we don't host the research, we just link it, & host the evaluations ,authors' responses, etc. (see unjournal.pubpub.org)

-->

The Unjournal

The Unjournal PubPub community: journal-independent evaluation of impactful work in economics, policy, and social science

unjournal.pubpub.org

1

The Unjournal (Unjournal.org) @unjournal.bsky.social · 13d

You also raise an important question:

1 1

The Unjournal (Unjournal.org) @unjournal.bsky.social · 13d

Each of our evaluations cites the original paper (as well as other papers).

But our evaluations do not tend to show up as "papers citing" the paper evaluated in Gscholar.

This limits a key path to visibility, which we have been struggling with.

The Unjournal (Unjournal.org) @unjournal.bsky.social · 13d

We also periodically add these to a server feeding repec.org.

This gets most of our content into Google Scholar, which is the tool most people seem to use.

But not always in an ordered way, and there's one main thing that we're missing:

RePEc: Research Papers in Economics

RePEc is a central index of economics research, including working papers, articles and software code

repec.org

1

The Unjournal (Unjournal.org) @unjournal.bsky.social · 13d

Looking forward to reading this. At unjournal.org, we use PubPub to mint a DOI for each of the evaluations of the papers that are evaluated, as well as the evaluation summary and authors' responses.

Unjournal | Transparent Evaluation of Impactful Research

The Unjournal enables open, expert evaluation of economics and social science research, without the constraints of traditional journals. Discover how we make rigorous research more transparent.

unjournal.org

1 1

The Unjournal (Unjournal.org) @unjournal.bsky.social · 16d

Thanks to @benbalmford.bsky.social Dr. Ben Balmford (www.benbalmford.com, LEEP, Exeter) for your strong work in this area, as well as to many diligent managers and evaluators.

www.exeter.ac.uk

The Unjournal (Unjournal.org) @unjournal.bsky.social · 16d

Climate change, environmental econ., biodiversity: Of the 50+ papers Unjournal.org commissioned for evaluation and assessment, ten focus on these areas, offering insight into methods & policy.

See our 'collection': https://bit.ly/4pKF0NT -->

Overall ratings by research/outcome area

Ratings for environment-related research

1

The Unjournal (Unjournal.org) @unjournal.bsky.social · 18d

Sorry, the top-4, funding 4 papers each, included
@j-pal.bsky.social

(IPA funded one paper we evaluated)

The Unjournal (Unjournal.org) @unjournal.bsky.social · 18d

2. To share info on which orgs tend to fund research judged to have the most potential for impact by The Unjournal team.

To help researchers & managers decide which institutions to work at, where to seek support, etc.

@80000hours.bsky.social @founderspledge.com ...

The Unjournal (Unjournal.org) @unjournal.bsky.social · 18d

We provide a faster & clearer feedback loop relative to traditional academic journals

1