News about our epistemic diversity paper!

💻 The code is now a python package! Installation instructions here: github.com/dwright37/ll...

🤗 All 1.6M model responses and 70M clustered claims are now available on HuggingFace! huggingface.co/datasets/dwr...

📄 Paper: arxiv.org/pdf/2510.04226

Heading to #EMNLP2025? Interested in automatic summarization? Then come to our poster for "Unstructured Evidence Attribution for Long Context Query Focused Summarization" !

⏰ When: Fri. Nov 7 14:00-15:30
🗺️ Where: Hall C

I'm unable to attend but @iaugenstein.bsky.social will present our work!

🌍 There are gaps in country specific knowledge. When matching claims to English and local language Wikipedia, no local language is statistically significantly more represented than English, and English language knowledge is statistically significantly more represented for 5 of 8 countries

6/10

🏗️ Model size has an unintuitive negative impact on diversity; smaller models tend to be more diverse

🔎 RAG has a positive impact on diversity, indicating its usefulness in making LLM outputs more diverse. However, the gains from RAG are not equal across topics about different countries

5/10

📈 Knowledge in LLMs across 3 of 4 model families has *expanded* since 2023 ✅ ; however, their absolute diversity is quite low compared to a very modest traditional search baseline 👎

4/10

Which, whose, and how much knowledge do LLMs represent?

I'm excited to share our preprint answering these questions:

"Epistemic Diversity and Knowledge Collapse in Large Language Models"

📄Paper: arxiv.org/pdf/2510.04226
💻Code: github.com/dwright37/ll...

1/10

🎉 Our work on attribution in summarization is now accepted to #EMNLP2025 main! 🎉

"Unstructured Evidence Attribution for Long Context Query Focused Summarization"

w/ @zainmujahid.me , Lu Wang, @iaugenstein.bsky.social , and @davidjurgens.bsky.social

There’s something really special about seeing a physical print copy of our work 🤩

You can read “Efficiency is Not Enough: A Critical Perspective on Environmentally Sustainable AI” now in CACM!!!

dl.acm.org/doi/10.1145/...

We had a great time talking about dialogue simulation with LLMs at @ic2s2.bsky.social !!! Amazing work by all of our colleagues at UMich.

See the preprint of this work here: arxiv.org/abs/2409.08330

SUnsET is created using a novel inductive synthetic data generation pipeline. Each stage has carefully engineered prompts designed to maximize diversity and accuracy. The dataset consists of documents broken into multiple sections, with multiple queries and summaries which cite specific text spans.

New preprint: "Unstructured Evidence Attribution for Long Context Query Focused Summarization"

We propose *unstructured* evidence attribution for long context summarization and a synthetic dataset called SUnsET which can help models perform this task.

Paper link: arxiv.org/abs/2502.14409

Thread ⬇️

Happening now @neuripsconf.bsky.social #NeurIPS2024 !!!

Come to poster 4003 in East Hall to learn about Bayesian model reduction for structured pruning!

I’m at #NeurIPS2024!

I’m also on the faculty job market!

Come talk to me about neural network efficiency, reliable NLP, open positions, and come to my spotlight poster on Friday from 11a — 2p in East Exhibit Hall A — C, poster number 4003

nominating @nadavb.bsky.social for our EMNLP poster

#EMNLP2024 was a blast!

Next stop is #NeurIPS2024 where I'll present our ✨spotlight✨ paper on thresholdless Bayesian structured pruning!

Paper: arxiv.org/pdf/2406.01345

Happening now at #EMNLP2024 ! Come to poster 34 in Jasmine to talk to me, @rnv.bsky.social and Srishti Yadav about LLM political values!

At #EMNLP2024 we will present our paper on LLM values and opinions!

We introduce tropes: repeated and consistent phrases which LLMs generate to argue for political stances.

Read the paper to learn more! arxiv.org/abs/2406.19238
Work done Uni Copenhagen + Pioneer Center for AI