David Duvenaud
@davidduvenaud.bsky.social
Machine learning prof at U Toronto. Working on evals and AGI governance.
How might the world look after the development of AGI, and what should we do about it now? Help us think about this at our workshop on Post-AGI Economics, Culture and Governance!
We’ll host speakers from political theory, economics, mechanism design, history, and hierarchical agency.
post-agi.org
We’ll host speakers from political theory, economics, mechanism design, history, and hierarchical agency.
post-agi.org
October 28, 2025 at 10:06 PM
How might the world look after the development of AGI, and what should we do about it now? Help us think about this at our workshop on Post-AGI Economics, Culture and Governance!
We’ll host speakers from political theory, economics, mechanism design, history, and hierarchical agency.
post-agi.org
We’ll host speakers from political theory, economics, mechanism design, history, and hierarchical agency.
post-agi.org
Me and Raymond Douglas on how AI job loss could hurt democracy. “No taxation without representation” summarizes that historically, democratic rights flow from economic power. But this might work in reverse once we’re all on UBI: No representation without taxation!
bsky.app/profile/econ...
bsky.app/profile/econ...
Without taxation there may be no representation, conclude Raymond Douglas and David Duvenaud
Two scholars ask whether democracy can survive if AI does all the jobs
Without taxation there may be no representation, conclude Raymond Douglas and David Duvenaud
econ.st
September 19, 2025 at 9:04 PM
Me and Raymond Douglas on how AI job loss could hurt democracy. “No taxation without representation” summarizes that historically, democratic rights flow from economic power. But this might work in reverse once we’re all on UBI: No representation without taxation!
bsky.app/profile/econ...
bsky.app/profile/econ...
It's hard to plan for AGI without knowing what outcomes are even possible, let alone good. So we’re hosting a workshop!
Post-AGI Civilizational Equilibria: Are there any good ones?
Vancouver, July 14th
www.post-agi.org
Featuring: Joe Carlsmith, @richardngo.bsky.social, Emmett Shear ... 🧵
Post-AGI Civilizational Equilibria: Are there any good ones?
Vancouver, July 14th
www.post-agi.org
Featuring: Joe Carlsmith, @richardngo.bsky.social, Emmett Shear ... 🧵
Post-AGI Civilizational Equilibria Workshop | Vancouver 2025
Are there any good ones? Join us in Vancouver on July 14th, 2025 to explore stable equilibria and human agency in a post-AGI world. Co-located with ICML.
www.post-agi.org
June 18, 2025 at 6:12 PM
It's hard to plan for AGI without knowing what outcomes are even possible, let alone good. So we’re hosting a workshop!
Post-AGI Civilizational Equilibria: Are there any good ones?
Vancouver, July 14th
www.post-agi.org
Featuring: Joe Carlsmith, @richardngo.bsky.social, Emmett Shear ... 🧵
Post-AGI Civilizational Equilibria: Are there any good ones?
Vancouver, July 14th
www.post-agi.org
Featuring: Joe Carlsmith, @richardngo.bsky.social, Emmett Shear ... 🧵
What to do about gradual disempowerment from AGI? We laid out a research agenda with all the concrete and feasible research projects we can think of: 🧵
www.lesswrong.com/posts/GAv4DR...
with Raymond Douglas, @kulveit.bsky.social @davidskrueger.bsky.social
www.lesswrong.com/posts/GAv4DR...
with Raymond Douglas, @kulveit.bsky.social @davidskrueger.bsky.social
Gradual Disempowerment: Concrete Research Projects — LessWrong
This post benefitted greatly from comments, suggestions, and ongoing discussions with David Duvenaud, David Krueger, and Jan Kulveit. All errors are…
www.lesswrong.com
June 3, 2025 at 9:22 PM
What to do about gradual disempowerment from AGI? We laid out a research agenda with all the concrete and feasible research projects we can think of: 🧵
www.lesswrong.com/posts/GAv4DR...
with Raymond Douglas, @kulveit.bsky.social @davidskrueger.bsky.social
www.lesswrong.com/posts/GAv4DR...
with Raymond Douglas, @kulveit.bsky.social @davidskrueger.bsky.social
Reposted by David Duvenaud
On top of the AISI-wide research agenda yesterday, we have more on the research agenda for the AISI Alignment Team specifically. See Benjamin's thread and full post for details; here I'll focus on why we should not give up on directly solving alignment, even though it is hard. 🧵
The Alignment Team at UK AISI now has a research agenda.
Our goal: solve the alignment problem.
How: develop concrete, parallelisable open problems.
Our initial focus is on asymptotic honesty guarantees (more details in the post).
1/5
Our goal: solve the alignment problem.
How: develop concrete, parallelisable open problems.
Our initial focus is on asymptotic honesty guarantees (more details in the post).
1/5
May 8, 2025 at 9:15 AM
On top of the AISI-wide research agenda yesterday, we have more on the research agenda for the AISI Alignment Team specifically. See Benjamin's thread and full post for details; here I'll focus on why we should not give up on directly solving alignment, even though it is hard. 🧵
Reposted by David Duvenaud
“What place will humans have when AI can do everything we do — only better?”
In The Guardian today, SRI Chair @davidduvenaud.bsky.social explores what happens when AI doesn't destroy us — it just quietly replaces us.
🔗 www.theguardian.com/books/2025/m...
#AI #AIEthics #TechAndSociety
In The Guardian today, SRI Chair @davidduvenaud.bsky.social explores what happens when AI doesn't destroy us — it just quietly replaces us.
🔗 www.theguardian.com/books/2025/m...
#AI #AIEthics #TechAndSociety
Better at everything: how AI could make human beings irrelevant
The end of civilisation might look less like a war, and more like a love story. Can we avoid being willing participants in our own downfall?
www.theguardian.com
May 5, 2025 at 5:59 PM
“What place will humans have when AI can do everything we do — only better?”
In The Guardian today, SRI Chair @davidduvenaud.bsky.social explores what happens when AI doesn't destroy us — it just quietly replaces us.
🔗 www.theguardian.com/books/2025/m...
#AI #AIEthics #TechAndSociety
In The Guardian today, SRI Chair @davidduvenaud.bsky.social explores what happens when AI doesn't destroy us — it just quietly replaces us.
🔗 www.theguardian.com/books/2025/m...
#AI #AIEthics #TechAndSociety
My single rule for productive Bluesky discussions:
Start every single reply with a point of agreement. It disarms the combative impulse on both sides, and forces you to try to interpret their words in the most sensible possible way.
Start every single reply with a point of agreement. It disarms the combative impulse on both sides, and forces you to try to interpret their words in the most sensible possible way.
March 19, 2025 at 8:49 PM
My single rule for productive Bluesky discussions:
Start every single reply with a point of agreement. It disarms the combative impulse on both sides, and forces you to try to interpret their words in the most sensible possible way.
Start every single reply with a point of agreement. It disarms the combative impulse on both sides, and forces you to try to interpret their words in the most sensible possible way.
New paper: What happens once AIs make humans obsolete?
Even without AIs seeking power, we argue that competitive pressures are set to fully erode human influence and values.
www.gradual-disempowerment.ai
with @kulveit.bsky.social, Raymond Douglas, Nora Ammann, Deger Turann, David Krueger 🧵
Even without AIs seeking power, we argue that competitive pressures are set to fully erode human influence and values.
www.gradual-disempowerment.ai
with @kulveit.bsky.social, Raymond Douglas, Nora Ammann, Deger Turann, David Krueger 🧵
January 30, 2025 at 5:19 PM
New paper: What happens once AIs make humans obsolete?
Even without AIs seeking power, we argue that competitive pressures are set to fully erode human influence and values.
www.gradual-disempowerment.ai
with @kulveit.bsky.social, Raymond Douglas, Nora Ammann, Deger Turann, David Krueger 🧵
Even without AIs seeking power, we argue that competitive pressures are set to fully erode human influence and values.
www.gradual-disempowerment.ai
with @kulveit.bsky.social, Raymond Douglas, Nora Ammann, Deger Turann, David Krueger 🧵
Happy to have helped a little with this paper:
New work from my team at Anthropic in collaboration with Redwood Research. I think this is plausibly the most important AGI safety result of the year. Cross-posting the thread below:
December 18, 2024 at 7:47 PM
Happy to have helped a little with this paper: