@ryantlowe.bsky.social
... and @davidduvenaud.bsky.social
@josephgubbels.bsky.social
@sydneylevine.bsky.social
@samuelemarro.bsky.social
@morgansutherland.bsky.social
@joelbot3000.bsky.social
@tobyshorin.bsky.social, among many many others :)
@josephgubbels.bsky.social
@sydneylevine.bsky.social
@samuelemarro.bsky.social
@morgansutherland.bsky.social
@joelbot3000.bsky.social
@tobyshorin.bsky.social, among many many others :)
July 11, 2025 at 7:11 PM
If you're excited about these ideas, drop me a line!! We're looking for researchers to collaborate with -- send an email to [email protected].
It's gonna be fun.
✌️
It's gonna be fun.
✌️
July 11, 2025 at 6:57 PM
If you're excited about these ideas, drop me a line!! We're looking for researchers to collaborate with -- send an email to [email protected].
It's gonna be fun.
✌️
It's gonna be fun.
✌️
This is a huge project. We'll need lots of help.
But if we succeed, the future could be more beautiful than we can possibly imagine today.
But if we succeed, the future could be more beautiful than we can possibly imagine today.
July 11, 2025 at 6:57 PM
This is a huge project. We'll need lots of help.
But if we succeed, the future could be more beautiful than we can possibly imagine today.
But if we succeed, the future could be more beautiful than we can possibly imagine today.
Examples of TMV include: resource-rational contractualism by @sydneymlevine et al, self-other overlap by
@MarcCarauleanu et al, and our previous work on moral graph elicitation.
It's an emerging field, but we think early research is very promising!!
@MarcCarauleanu et al, and our previous work on moral graph elicitation.
It's an emerging field, but we think early research is very promising!!
July 11, 2025 at 6:57 PM
Examples of TMV include: resource-rational contractualism by @sydneymlevine et al, self-other overlap by
@MarcCarauleanu et al, and our previous work on moral graph elicitation.
It's an emerging field, but we think early research is very promising!!
@MarcCarauleanu et al, and our previous work on moral graph elicitation.
It's an emerging field, but we think early research is very promising!!
Instead, we call for a new paradigm — "Thick models of value" (TMV).
TMV is a broad class of structured approaches to modeling values and norms that:
1. are more robust against distortions
2. have better treatment of collective values and norms
3. have better generalization
TMV is a broad class of structured approaches to modeling values and norms that:
1. are more robust against distortions
2. have better treatment of collective values and norms
3. have better generalization
July 11, 2025 at 6:57 PM
Instead, we call for a new paradigm — "Thick models of value" (TMV).
TMV is a broad class of structured approaches to modeling values and norms that:
1. are more robust against distortions
2. have better treatment of collective values and norms
3. have better generalization
TMV is a broad class of structured approaches to modeling values and norms that:
1. are more robust against distortions
2. have better treatment of collective values and norms
3. have better generalization
It's like programming in a completely dynamic language without any type system. Kinda sketch if we design all of our institutions around unstructured text and hope LLMs interpret them in the way we intended.
July 11, 2025 at 6:56 PM
It's like programming in a completely dynamic language without any type system. Kinda sketch if we design all of our institutions around unstructured text and hope LLMs interpret them in the way we intended.
In principle, unstructured text could be an improvement; after all, language is how humans naturally express values.
But this lack of internal structure becomes a critical weakness when we need *reliability* across contexts and institutions.
But this lack of internal structure becomes a critical weakness when we need *reliability* across contexts and institutions.
July 11, 2025 at 6:56 PM
In principle, unstructured text could be an improvement; after all, language is how humans naturally express values.
But this lack of internal structure becomes a critical weakness when we need *reliability* across contexts and institutions.
But this lack of internal structure becomes a critical weakness when we need *reliability* across contexts and institutions.
In practice this looks like: a desire for "meaningful connection" becomes "engagement metrics" to recommender systems, which becomes "daily active users" to companies, and "quarterly revenue" in markets.
July 11, 2025 at 6:56 PM
In practice this looks like: a desire for "meaningful connection" becomes "engagement metrics" to recommender systems, which becomes "daily active users" to companies, and "quarterly revenue" in markets.
PMV in particular (the dominant paradigm in microeconomics, game theory, mechanism design, social choice theory, etc) fails to capture the richness of human motivation, because preferences bundle all kinds of signals into a flattened ordering.
July 11, 2025 at 6:56 PM
PMV in particular (the dominant paradigm in microeconomics, game theory, mechanism design, social choice theory, etc) fails to capture the richness of human motivation, because preferences bundle all kinds of signals into a flattened ordering.
Current approaches tend to fall into what we call "Preferentist models of Value" (PMV), or "Values-as-text" (VAT). Both have issues preserving the richness of what people care about, as value information propagates up the "societal stack"
July 11, 2025 at 6:56 PM
Current approaches tend to fall into what we call "Preferentist models of Value" (PMV), or "Values-as-text" (VAT). Both have issues preserving the richness of what people care about, as value information propagates up the "societal stack"
Okay, so this seems extremely hard. How do we do it?
We think we've identified an important piece of the puzzle: we need better *representations* of values and norms that are legible to AI, markets and democracies.
We think we've identified an important piece of the puzzle: we need better *representations* of values and norms that are legible to AI, markets and democracies.
July 11, 2025 at 6:56 PM
Okay, so this seems extremely hard. How do we do it?
We think we've identified an important piece of the puzzle: we need better *representations* of values and norms that are legible to AI, markets and democracies.
We think we've identified an important piece of the puzzle: we need better *representations* of values and norms that are legible to AI, markets and democracies.
Important to say: the goal of full-stack alignment is pluralistic. We don't impose a singular vision of human flourishing.
Rather, we argue that we need better institutional tools to improve our capacity for individual and collective self-determination.
Rather, we argue that we need better institutional tools to improve our capacity for individual and collective self-determination.
July 11, 2025 at 6:56 PM
Important to say: the goal of full-stack alignment is pluralistic. We don't impose a singular vision of human flourishing.
Rather, we argue that we need better institutional tools to improve our capacity for individual and collective self-determination.
Rather, we argue that we need better institutional tools to improve our capacity for individual and collective self-determination.
So, full-stack alignment is our way of saying that we need AI and institutions that "fit" us, and that help us live the lives we want to live.
July 11, 2025 at 6:56 PM
So, full-stack alignment is our way of saying that we need AI and institutions that "fit" us, and that help us live the lives we want to live.
We have markets that favor products that are addictive and isolating, and our democratic institutions are hyper-polarizing us.
Things will get worse as AI displaces workers and out-paces regulation. In our near-term future we risk losing both our economic and democratic agency.
Things will get worse as AI displaces workers and out-paces regulation. In our near-term future we risk losing both our economic and democratic agency.
July 11, 2025 at 6:56 PM
We have markets that favor products that are addictive and isolating, and our democratic institutions are hyper-polarizing us.
Things will get worse as AI displaces workers and out-paces regulation. In our near-term future we risk losing both our economic and democratic agency.
Things will get worse as AI displaces workers and out-paces regulation. In our near-term future we risk losing both our economic and democratic agency.
Why do we need to co-align AI *and* institutions?
AI systems don't exist in a vacuum. They are embedded within institutions whose incentives shape their deployment.
Often, institutional incentives are not aligned with what's in our best interest.
AI systems don't exist in a vacuum. They are embedded within institutions whose incentives shape their deployment.
Often, institutional incentives are not aligned with what's in our best interest.
July 11, 2025 at 6:56 PM
Why do we need to co-align AI *and* institutions?
AI systems don't exist in a vacuum. They are embedded within institutions whose incentives shape their deployment.
Often, institutional incentives are not aligned with what's in our best interest.
AI systems don't exist in a vacuum. They are embedded within institutions whose incentives shape their deployment.
Often, institutional incentives are not aligned with what's in our best interest.
This is in collaboration with a huge list of really really amazing researchers (see the list of authors!)
July 11, 2025 at 6:56 PM
This is in collaboration with a huge list of really really amazing researchers (see the list of authors!)
Today we're launching:
- A position paper that articulates the conceptual foundations of FSA (jytmawd4y4kxsznl.public.blob.vercel-storage.com/Full_Stack_...)
- A website which will be the homepage of FSA going forward (www.full-stack-alignment.ai/)
- A position paper that articulates the conceptual foundations of FSA (jytmawd4y4kxsznl.public.blob.vercel-storage.com/Full_Stack_...)
- A website which will be the homepage of FSA going forward (www.full-stack-alignment.ai/)
July 11, 2025 at 6:56 PM
Today we're launching:
- A position paper that articulates the conceptual foundations of FSA (jytmawd4y4kxsznl.public.blob.vercel-storage.com/Full_Stack_...)
- A website which will be the homepage of FSA going forward (www.full-stack-alignment.ai/)
- A position paper that articulates the conceptual foundations of FSA (jytmawd4y4kxsznl.public.blob.vercel-storage.com/Full_Stack_...)
- A website which will be the homepage of FSA going forward (www.full-stack-alignment.ai/)
Reposted
And: I'm hiring a post-doc! I'm looking for someone with a strong computational background who can start in the summer (or sooner). Details here: apply.interfolio.com/165122
Feel free to reach out with questions (email is best: [email protected]) and please share widely!
Feel free to reach out with questions (email is best: [email protected]) and please share widely!
April 7, 2025 at 11:43 PM
And: I'm hiring a post-doc! I'm looking for someone with a strong computational background who can start in the summer (or sooner). Details here: apply.interfolio.com/165122
Feel free to reach out with questions (email is best: [email protected]) and please share widely!
Feel free to reach out with questions (email is best: [email protected]) and please share widely!