Lightnews — Scholar-powered news

Reposted by Elinor🎗️ @ COLM 🍁

Josh Moody @byjoshmoody.bsky.social · 5d

MIT rejects "compact" proposed by the Trump administration.
MIT prez wrote: it "would restrict freedom of expression and our independence as an institution" and "is inconsistent with our core belief that scientific funding should be based on scientific merit alone."
orgchart.mit.edu/letters/rega...

Regarding the Compact | MIT Organization Chart

orgchart.mit.edu

34 490 1.9K

Reposted by Elinor🎗️ @ COLM 🍁

Hope Schroeder @hopeschroeder.bsky.social · 5d

Hello #COLM2025! Excited to be kicking off the NLP4Democracy workshop this morning. We are in 520E (behind A/B/C) - check out our amazing program! sites.google.com/andrew.cmu.e...

NLP 4 Democracy - COLM 2025

sites.google.com

1 4

Reposted by Elinor🎗️ @ COLM 🍁

Valentina Pyatkin @valentinapy.bsky.social · 5d

💡We kicked off the SoLaR workshop at #COLM2025 with a great opinion talk by @michelleding.bsky.social & Jo Gasior Kavishe (joint work with @victorojewale.bsky.social and
@geomblog.bsky.social
) on "Testing LLMs in a sandbox isn't responsible. Focusing on community use and needs is."

1 4 14

Reposted by Elinor🎗️ @ COLM 🍁

Michelle L. Ding @michelleding.bsky.social · 5d

Hi #COLM2025! 🇨🇦 I will be presenting a talk on the importance of community-driven LLM evaluations based on an opinion abstract I wrote with Jo Kavishe, @victorojewale.bsky.social and @geomblog.bsky.social tomorrow at 9:30am in 524b for solar-colm.github.io

Hope to see you there!

Third Workshop on Socially Responsible Language Modelling Research (SoLaR) 2025

COLM 2025 in-person Workshop, October 10th at the Palais des Congrès in Montreal, Canada

solar-colm.github.io

1 6 9

Elinor🎗️ @ COLM 🍁 @elinorpd.bsky.social · 5d

adding on, he also said "We have a responsibility to make sure we have a good answer to this question" in the field

1

Elinor🎗️ @ COLM 🍁 @elinorpd.bsky.social · 5d

bsky.app/profile/mari...

Slide with “we are here” in bottom left with an arrow pointing top right towards two groups of words: “AI2027” & “everyone dies” and then “AI con” “AI snake oil” “Empire of AI”

2 6

Elinor🎗️ @ COLM 🍁 @elinorpd.bsky.social · 5d

Important keynote by Nicholas Carlini with important calls to action for the research community!

Ty for the helpful summary @mariaa.bsky.social

Maria Antoniak @mariaa.bsky.social · 6d

"What problems you're scared of depend on how good you think the LLMs will get"

"Please be willing to change your mind."

"This is COLM. We made the models, it's our job to fix it. How are you going to change your research agenda?"

#COLM2025

1 1 6

Reposted by Elinor🎗️ @ COLM 🍁

David Mimno @dmimno.bsky.social · 8d

COLM word cloud. Yoav says it’s the year of reasoning, but evaluation is also huge.

Evaluation reasoning interpretability rl in context benchmark alignment synthetic data

3 5 30

Elinor🎗️ @ COLM 🍁 @elinorpd.bsky.social · 8d

bsky.app/profile/elin...

Elinor🎗️ @ COLM 🍁 @elinorpd.bsky.social · 27d

🚨 New preprint! 🚨
Excited to share my work: An AI-Powered Framework for Analyzing Collective Idea Evolution in Deliberative Assemblies 🤖🗳️

I’ll be presenting this at @colmweb.org in the NLP4Democracy workshop!

🔗 arxiv.org/abs/2509.12577

An AI-Powered Framework for Analyzing Collective Idea Evolution in Deliberative Assemblies

In an era of increasing societal fragmentation, political polarization, and erosion of public trust in institutions, representative deliberative assemblies are emerging as a promising democratic forum...

arxiv.org

1

Elinor🎗️ @ COLM 🍁 @elinorpd.bsky.social · 8d

I’m at #COLM2025! Would love to chat about anything related to pluralistic alignment, fairness evaluations, societal impacts of LLMs, etc 😊

You can also find me at the NLP4Democracy workshop giving a talk about my work analyzing democratic deliberation with LLMs Oct 10th!

1 2

Reposted by Elinor🎗️ @ COLM 🍁

No Score Draws ✍🏻⚽ @noscoredraws.com · 8d

Alright the evening sky, you’re utterly wondrous and fantastical, we get it, geez

4 11 220

Reposted by Elinor🎗️ @ COLM 🍁

Maria Antoniak @mariaa.bsky.social · 8d

Here’s a #COLM2025 feed!

Pin it 📌 to follow along with the conference this week!

2 17 26

Reposted by Elinor🎗️ @ COLM 🍁

Julia Mendelsohn @jmendelsohn2.bsky.social · 8d

I will be at #COLM2025 this week, and would love to connect with folks interested in applications (and critiques) of language modeling in social science research!

And join us for the NLP4Democracy workshop on Friday!

sites.google.com/andrew.cmu.e...

#NLP #NLProc #LLM #ComputationalSocialScience

NLP 4 Democracy - COLM 2025

sites.google.com

5 16

Reposted by Elinor🎗️ @ COLM 🍁

Andy Halterman @ahalterman.bsky.social · 26d

Very excited that my paper with @katakeith.bsky.social is now out in @polanalysis.bsky.social. We investigate whether LLMs actually follow the instructions/definitions provided in codebooks, propose some diagnostics, and release a new evaluation dataset.
www.cambridge.org/core/journal...

Codebook LLMs: Evaluating LLMs as Measurement Tools for Political Science Concepts | Political Analysis | Cambridge Core

Codebook LLMs: Evaluating LLMs as Measurement Tools for Political Science Concepts

www.cambridge.org

14 31

Reposted by Elinor🎗️ @ COLM 🍁

Naomi Saphra @nsaphra.bsky.social · 25d

I wish students understood in most empirical AI research there’s a huge scientific advantage from being constitutionally excited by math vs intimidated, but very little additional gain from being actually “good” at math. Maybe they’d be less intimidated if they didn’t feel they had to be “good”.

7 4 51

Elinor🎗️ @ COLM 🍁 @elinorpd.bsky.social · 27d

This work is part of my master’s thesis at @mit.edu @medialab.bsky.social, supervised by Deb Roy and with the help of Jad Kabbara @jad-kabbara.bsky.social.

🔗 arxiv.org/abs/2509.12577

An AI-Powered Framework for Analyzing Collective Idea Evolution in Deliberative Assemblies

In an era of increasing societal fragmentation, political polarization, and erosion of public trust in institutions, representative deliberative assemblies are emerging as a promising democratic forum...

arxiv.org

1

Elinor🎗️ @ COLM 🍁 @elinorpd.bsky.social · 27d

Beyond research, this paves the way for:
✨ Tools supporting live assemblies in real time
✨ Increasing transparency & communicating critical insights to decision-makers
✨ Enabling richer cross-assembly analysis to advance research on deliberative best practices

1

Elinor🎗️ @ COLM 🍁 @elinorpd.bsky.social · 27d

In the tech-enhanced assembly, our framework revealed:
🔹 How deliberation surfaced, refined, or discarded ideas
🔹 *Missing* viable ideas
🔹 How opinion shifts & rec edits shaped outcomes
🔹 Underlying values & trade-offs invisible to decision-makers

1

Elinor🎗️ @ COLM 🍁 @elinorpd.bsky.social · 27d

We develop an LLM-based framework to:
✅ Map how suggestions transform into concrete recommendations
✅ Reconstruct individuals’ evolving perspectives
✅ Detect why votes shift across deliberation

1

Elinor🎗️ @ COLM 🍁 @elinorpd.bsky.social · 27d

Despite their promise, we still lack tools to empirically trace:
• how ideas evolve into recommendations
• how deliberation shapes perspectives & votes

At MIT CCC, we hosted our own tech-enhanced assembly to explore how AI can help!

sustainabilityassembly.portal.cortico.ai

Loading...

sustainabilityassembly.portal.cortico.ai

1

Elinor🎗️ @ COLM 🍁 @elinorpd.bsky.social · 27d

Deliberative assemblies bring together everyday citizens selected by lottery. Through deliberation 💬 & learning, they collectively form policy recommendations 💡for decision-makers.

They’ve proven successful worldwide, facilitating rebuilding trust & strengthening democracy 🤝.

1 1

Elinor🎗️ @ COLM 🍁 @elinorpd.bsky.social · 27d

🚨 New preprint! 🚨
Excited to share my work: An AI-Powered Framework for Analyzing Collective Idea Evolution in Deliberative Assemblies 🤖🗳️

I’ll be presenting this at @colmweb.org in the NLP4Democracy workshop!

🔗 arxiv.org/abs/2509.12577

An AI-Powered Framework for Analyzing Collective Idea Evolution in Deliberative Assemblies

In an era of increasing societal fragmentation, political polarization, and erosion of public trust in institutions, representative deliberative assemblies are emerging as a promising democratic forum...

arxiv.org

1 2 7

Elinor🎗️ @ COLM 🍁 @elinorpd.bsky.social · 29d

"This suggests that LLM benchmark behavior may generalize less and less to non-benchmark settings, raising new concerns about ecological validity."

super interesting

Bufan Gao @bufangao.bsky.social · Sep 11

🚨 New #EMNLP2025 paper!

Do LLMs exhibit distinct behavior when the prompt looks similar to common evaluation prompts? 👀

We show that prompts that signal bias evaluation can flip the measured bias. See below ⬇️

Violin plots of Probability of Pronoun Shift. Models show significant sensitivity to prompt changes: when prompts highlight gender evaluation, pronoun use shifts, with decreased “he” and increased “they” use.

1

Reposted by Elinor🎗️ @ COLM 🍁

Yoshua Bengio @yoshuabengio.bsky.social · Sep 15

This paper yields the same conclusion as what @mustafasuleymanai.bsky.social recently posted on the danger of 'seemingly conscious AI'.

mustafa-suleyman.ai/seemingly-co...

We must build AI for people; not to be a person

mustafa-suleyman.ai

4 15

Elinor🎗️ @ COLM 🍁 @elinorpd.bsky.social · Sep 14

Thread inspired by having to review 6(!!) papers for AAAI and most of them having no line numbers. And one particularly great paper I want to show the authors exactly how much I enjoyed it via my annotation drawings (>20 check marks, ~10 exclamations, and even 2 hearts!)

3