Lightnews — Scholar-powered news

Atoosa Kasirzadeh @atoosakz.bsky.social · 4h

Thanks, Stephen! BTW, the last version of the paper is here and open access: link.springer.com/article/10.1...

I recommend reading that

Two types of AI existential risk: decisive and accumulative - Philosophical Studies

The conventional discourse on existential risks (x-risks) from AI typically focuses on abrupt, dire events caused by advanced AI systems, particularly those that might achieve or surpass human-level i...

link.springer.com

1 2

Reposted by Atoosa Kasirzadeh

bleary @bleary.off-the-records.com · 1d

If anyone needs me I will be in the museum, lying down next to the bog bodies.

Did people really memorize phone numbers before cell phones, or is that just a movie thing?
2? Questions
I was watching some old shows from the 90s and noticed people would just dial numbers from memory - like they'd call their friends or family without looking anything up.
Made me wonder if that was actually normal back then? Did people genuinely have all their important numbers memorized, or did most folks keep a little address book or written list nearby?

1.5K 4.8K 23K

Atoosa Kasirzadeh @atoosakz.bsky.social · 3d

Huge congratulations, Aylin!!!

1

Reposted by Atoosa Kasirzadeh

Max Kleiman-Weiner @maxkw.bsky.social · 12d

Great work led by @andyliu.bsky.social and collaborators:
@kghate.bsky.social, @monadiab77.bsky.social, @daniel-fried.bsky.social, @atoosakz.bsky.social
Preprint: www.arxiv.org/abs/2509.25369

Generative Value Conflicts Reveal LLM Priorities

Past work seeks to align large language model (LLM)-based assistants with a target set of values, but such assistants are frequently forced to make tradeoffs between values when deployed. In response ...

www.arxiv.org

1 3

Reposted by Atoosa Kasirzadeh

Andy Liu @andyliu.bsky.social · 12d

🚨New Paper: LLM developers aim to align models with values like helpfulness or harmlessness. But when these conflict, which values do models choose to support? We introduce ConflictScope, a fully-automated evaluation pipeline that reveals how models rank values under conflict.
(📷 xkcd)

1 4 13

Reposted by Atoosa Kasirzadeh

Bennett @bennettmcintosh.com · 27d

If you read one review of If Anyone Builds It Everyone Dies make it @sigalsamuel.bsky.social 's about competing #AI worldviews

"It was hard to seriously entertain both [doomer and AI-as-normal tech] views at the same time."

www.vox.com/future-perfe...

The AI doomers are not making an argument. They’re selling a worldview.

How rational is Eliezer Yudkowsky’s prophecy?

www.vox.com

1 4 4

Reposted by Atoosa Kasirzadeh

Sigal Samuel @sigalsamuel.bsky.social · 26d

I had a very trippy experience reading the new Eliezer Yudkowsky book, IF ANYONE BUILDS IT, EVERYONE DIES.

I agree with the basic idea that the current speed & trajectory of AI progress is incredibly dangerous!

But I don't buy his general worldview. Here's why:

www.vox.com/future-perfe...

“AI will kill everyone” is not an argument. It’s a worldview.

How rational is Eliezer Yudkowsky’s prophecy?

www.vox.com

1 2 6

Reposted by Atoosa Kasirzadeh

Lilian Edwards @lilianedwards.bsky.social · 21d

Very happy to announce brilliant new paper on AI by the wonderful @atoosakz.bsky.social , Phillip Hacker, and er, me, on systemic risk and how it is confusingly differently interpreted through the EU AIA, the DSA and its origin, financial regulation.

Paper at https://arxiv.org/pdf/2509.17878

1 4 18

Reposted by Atoosa Kasirzadeh

Bennett @bennettmcintosh.com · 27d

Kasirzadeh also has a really insightful, heartbreaking thread about what "normal" hides here: bsky.app/profile/atoo...

Atoosa Kasirzadeh @atoosakz.bsky.social · Jun 15

Normal isn't a fact; it's a perception built from repetition & power. My brother can call a night of missile fire normal simply because he's used to it.To an outsider, the scene is clearly abnormal. Judgements of normality rest on thin, shifting ground. History shows how quickly baselines move. 5/n

1 1

Reposted by Atoosa Kasirzadeh

Bennett @bennettmcintosh.com · 27d

Anyway, big fan of the 3rd option Sigal presents, @atoosakz.bsky.social 's warning of "gradual accumulation of smaller, seemingly non-existential, AI risks" until catastrophe.

A warning that suggests AI safetyists should take AI ethics & sociology much more seriously! arxiv.org/pdf/2401.07836

arxiv.org

1 2 3

Atoosa Kasirzadeh @atoosakz.bsky.social · Sep 11

How good are the current AI agents for scientific discovery? We have answers in our new paper: lnkd.in/dxzmZXpR!
We look at 4 ways AI scientists can go wrong; design experiments to show the manifestation of these failures in 2 open source AI scientists; and recommend detection strategies.

1 6

Atoosa Kasirzadeh @atoosakz.bsky.social · Aug 19

We are very excited to have you, Bálint!

2

Atoosa Kasirzadeh @atoosakz.bsky.social · Aug 19

🔊📢 Call for paper is now open (iaseai.org/iaseai26)! I’m thrilled to share that I will once again be serving as Program Co-Chair for International Association for Safe & Ethical AI Conference, this time for the 2026 edition along with the Co-Chair Jaime Fernández Fisac.

IASEAI'26

Building a Global Movement for Safe and Ethical AI

iaseai.org

1 2

Reposted by Atoosa Kasirzadeh

ryantlowe.bsky.social @ryantlowe.bsky.social · Jul 11

Introducing: Full-Stack Alignment 🥞

A research program dedicated to co-aligning AI systems *and* institutions with what people value.

It's the most ambitious project I've ever undertaken.

Here's what we're doing: 🧵

1 7 16

Atoosa Kasirzadeh @atoosakz.bsky.social · Jul 10

At hashtag#AIforGood Summit 2025: ‘AI agents’ is mentioned almost every panel, from law and governance to the developer community; yet the term remains so opaque. A plug in to our paper with Iason Gabriel which offers one of the clearest definition I’ve seen: arxiv.org/pdf/2504.21848

arxiv.org

4

Atoosa Kasirzadeh @atoosakz.bsky.social · Jun 16

I was planning to launch my substack on "Human, life, AI, and future" in a few months, with something very different. But life doesn’t always care about our timelines. Events erupt. Emotions build. And suddenly, waiting feels like avoidance. My first post: atoosatopia.substack.com/p/valid-and-...

1 1

Atoosa Kasirzadeh @atoosakz.bsky.social · Jun 15

You can read AI as normal technology here: knightcolumbia.org/content/ai-a...

1

Atoosa Kasirzadeh @atoosakz.bsky.social · Jun 15

But because judgments of normal are too subjective—and because the world we live in entangled with advanved AI systems is anything but normal—the metaphor can conceals more than it reveals. So here is my proposal: Keep the demystification project live; drop the label “normal”. 10/n

1

Atoosa Kasirzadeh @atoosakz.bsky.social · Jun 15

—I’ve pushed back against the “super-intelligence will kill us all” narrative myself for years.
(See for example: Two types of AI existential risk: decisive and accumulative: link.springer.com/article/10.1... AI safety for everyone: www.nature.com/articles/s42...) 9/n

2 1 3

Atoosa Kasirzadeh @atoosakz.bsky.social · Jun 15

A system that decides who lives or dies is nothing like a household appliance. I admire the scholars Arvind Narayanan and Sayash Kapoor who coined the phrase and share their wish to redefine the notion of “normal” to move the AI governance debate beyond extreme doom. 8/n

This link will take you to a page that’s not on LinkedIn

lnkd.in

1

Atoosa Kasirzadeh @atoosakz.bsky.social · Jun 15

The metaphor of "AI as normal technology" appeals cause it demystifies in the first place. But the label normal collapses the moment AI directs drones, misidentifies civilians, rewrites battlefield doctrine, or nudges users toward self-harm. 7/n

1

Atoosa Kasirzadeh @atoosakz.bsky.social · Jun 15

Seat belts were fussy in 1960, masks were not normal in 2019; both now signal basic care. Those who control information circulation or policy can freeze and stabilize a narrative in place, branding a live hazard normal by sheer repetition. 6/n

1

Atoosa Kasirzadeh @atoosakz.bsky.social · Jun 15

Normal isn't a fact; it's a perception built from repetition & power. My brother can call a night of missile fire normal simply because he's used to it.To an outsider, the scene is clearly abnormal. Judgements of normality rest on thin, shifting ground. History shows how quickly baselines move. 5/n

1

Atoosa Kasirzadeh @atoosakz.bsky.social · Jun 15

A toaster browns bread & stops; an agentic chatbot can keep learning and might suddenly write ransomware. Consumer devices have fixed functions & well-tested safety rules. Self-modifying agentic systems don't. Using the same label 4 both narrows the questions we ask about oversight & liability. 4/n

1

Atoosa Kasirzadeh @atoosakz.bsky.social · Jun 15

When most people hear "normal technology" I bet they picture safe, familiar gadgets—kettles, routers, maybe a smartphone. They don't picture autonomous drones, facial-recognition gunsights, or psychosis chatbots. The "normal" metaphor understates the scale of impact that modern AI can inflict. 3/n

1