Cas (Stephen Casper)
@scasper.bsky.social
AI technical gov & risk management research. PhD student @MIT_CSAIL, fmr. UK AISI. I'm on the CS faculty job market! https://stephencasper.com/
Here are my current favorite ideas for how to improve tamper-resistant ignorance/unlearning in LLMs.
Shamelessly copied from a slack message.
Shamelessly copied from a slack message.
November 26, 2025 at 4:00 PM
Here are my current favorite ideas for how to improve tamper-resistant ignorance/unlearning in LLMs.
Shamelessly copied from a slack message.
Shamelessly copied from a slack message.
November 25, 2025 at 8:00 PM
In general, it's still hard to study the impacts of data filtering because the experiments are expensive, & developers don't generally report much about what they do. For example, we found very limited/inconsistent reporting in some recent analysis.
t.co/CVkAKNXZme
t.co/CVkAKNXZme
November 25, 2025 at 8:00 PM
In general, it's still hard to study the impacts of data filtering because the experiments are expensive, & developers don't generally report much about what they do. For example, we found very limited/inconsistent reporting in some recent analysis.
t.co/CVkAKNXZme
t.co/CVkAKNXZme
5. Biorisk evals paper (Nov 2025)
They tested filtration of species/genus data against adv. fine-tuning. It didn't work well. This suggests filtering may work better if applied to entire tasks/domains rather than specific instances.
arxiv.org/abs/2510.27629
They tested filtration of species/genus data against adv. fine-tuning. It didn't work well. This suggests filtering may work better if applied to entire tasks/domains rather than specific instances.
arxiv.org/abs/2510.27629
November 25, 2025 at 8:00 PM
5. Biorisk evals paper (Nov 2025)
They tested filtration of species/genus data against adv. fine-tuning. It didn't work well. This suggests filtering may work better if applied to entire tasks/domains rather than specific instances.
arxiv.org/abs/2510.27629
They tested filtration of species/genus data against adv. fine-tuning. It didn't work well. This suggests filtering may work better if applied to entire tasks/domains rather than specific instances.
arxiv.org/abs/2510.27629
4. Deep ignorance paper (August 2025) @kyletokens.bsky.social
We showed that filtering biothreat-related pretraining data is SOTA for making models resist adversarial fine-tuning. We proposed an amendment to the hypothesis from papers 1 and 2 above.
deepignorance.ai
We showed that filtering biothreat-related pretraining data is SOTA for making models resist adversarial fine-tuning. We proposed an amendment to the hypothesis from papers 1 and 2 above.
deepignorance.ai
November 25, 2025 at 8:00 PM
4. Deep ignorance paper (August 2025) @kyletokens.bsky.social
We showed that filtering biothreat-related pretraining data is SOTA for making models resist adversarial fine-tuning. We proposed an amendment to the hypothesis from papers 1 and 2 above.
deepignorance.ai
We showed that filtering biothreat-related pretraining data is SOTA for making models resist adversarial fine-tuning. We proposed an amendment to the hypothesis from papers 1 and 2 above.
deepignorance.ai
3. Estimating worst-case open-weight risks paper (Aug 2025)
They reported an instance where filtering biothreat data didn't have a big impact. But without more info on how and how much they filtered, it's hard to draw strong conclusions.
arxiv.org/abs/2508.03153
They reported an instance where filtering biothreat data didn't have a big impact. But without more info on how and how much they filtered, it's hard to draw strong conclusions.
arxiv.org/abs/2508.03153
November 25, 2025 at 8:00 PM
3. Estimating worst-case open-weight risks paper (Aug 2025)
They reported an instance where filtering biothreat data didn't have a big impact. But without more info on how and how much they filtered, it's hard to draw strong conclusions.
arxiv.org/abs/2508.03153
They reported an instance where filtering biothreat data didn't have a big impact. But without more info on how and how much they filtered, it's hard to draw strong conclusions.
arxiv.org/abs/2508.03153
2. Bad data --> good models paper (May 2025)
They found similar results to the safety pretraining paper -- that models trained on without toxic text could be *more* vulnerable to attacks eliciting toxicity.
arxiv.org/abs/2505.04741
They found similar results to the safety pretraining paper -- that models trained on without toxic text could be *more* vulnerable to attacks eliciting toxicity.
arxiv.org/abs/2505.04741
November 25, 2025 at 8:00 PM
2. Bad data --> good models paper (May 2025)
They found similar results to the safety pretraining paper -- that models trained on without toxic text could be *more* vulnerable to attacks eliciting toxicity.
arxiv.org/abs/2505.04741
They found similar results to the safety pretraining paper -- that models trained on without toxic text could be *more* vulnerable to attacks eliciting toxicity.
arxiv.org/abs/2505.04741
1. Safety pretraining paper (Apr 2025)
The experiment of theirs that was most interesting to me found that models trained without toxic text could be *more* vulnerable to attacks eliciting toxicity.
arxiv.org/abs/2504.16980
The experiment of theirs that was most interesting to me found that models trained without toxic text could be *more* vulnerable to attacks eliciting toxicity.
arxiv.org/abs/2504.16980
November 25, 2025 at 8:00 PM
1. Safety pretraining paper (Apr 2025)
The experiment of theirs that was most interesting to me found that models trained without toxic text could be *more* vulnerable to attacks eliciting toxicity.
arxiv.org/abs/2504.16980
The experiment of theirs that was most interesting to me found that models trained without toxic text could be *more* vulnerable to attacks eliciting toxicity.
arxiv.org/abs/2504.16980
The leaked executive order has me wondering if the term "regulatory capture" has any meaning anymore.
It appears that state AI bills -- many of which big tech has fought tooth and nail to prevent -- are categorically regulatory capture.
It appears that state AI bills -- many of which big tech has fought tooth and nail to prevent -- are categorically regulatory capture.
November 20, 2025 at 2:00 PM
The leaked executive order has me wondering if the term "regulatory capture" has any meaning anymore.
It appears that state AI bills -- many of which big tech has fought tooth and nail to prevent -- are categorically regulatory capture.
It appears that state AI bills -- many of which big tech has fought tooth and nail to prevent -- are categorically regulatory capture.
We also find that currently, prominent open-weight model developers often either do not implement or report on mitigations. So there is a lot of room for more innovation and information as the science grows.
November 12, 2025 at 2:17 PM
We also find that currently, prominent open-weight model developers often either do not implement or report on mitigations. So there is a lot of room for more innovation and information as the science grows.
Empirical harms enabled by open models are also mounting. For example, the Internet Watch Foundation has found that they are the tools of choice for generating non-consensual AI deepfakes depicting children.
t.co/Ag4J6rrejz
t.co/Ag4J6rrejz
November 12, 2025 at 2:17 PM
Empirical harms enabled by open models are also mounting. For example, the Internet Watch Foundation has found that they are the tools of choice for generating non-consensual AI deepfakes depicting children.
t.co/Ag4J6rrejz
t.co/Ag4J6rrejz
Most importantly, powerful open-weight models are probably inevitable. For example, in recent years, they have steadily grown in their prominence, capabilities, and influence. Here are two nice graphics I often point to.
Thx Epoch & Bhandari et al.
Thx Epoch & Bhandari et al.
November 12, 2025 at 2:17 PM
Most importantly, powerful open-weight models are probably inevitable. For example, in recent years, they have steadily grown in their prominence, capabilities, and influence. Here are two nice graphics I often point to.
Thx Epoch & Bhandari et al.
Thx Epoch & Bhandari et al.
🚨New paper🚨
From a technical perspective, safeguarding open-weight model safety is AI safety in hard mode. But there's still a lot of progress to be made. Our new paper covers 16 open problems.
🧵🧵🧵
From a technical perspective, safeguarding open-weight model safety is AI safety in hard mode. But there's still a lot of progress to be made. Our new paper covers 16 open problems.
🧵🧵🧵
November 12, 2025 at 2:17 PM
🚨New paper🚨
From a technical perspective, safeguarding open-weight model safety is AI safety in hard mode. But there's still a lot of progress to be made. Our new paper covers 16 open problems.
🧵🧵🧵
From a technical perspective, safeguarding open-weight model safety is AI safety in hard mode. But there's still a lot of progress to be made. Our new paper covers 16 open problems.
🧵🧵🧵
We also find that currently, prominent open-weight model developers often either do not implement or report on mitigations. So there is a lot of room for more innovation and information as the science grows.
November 12, 2025 at 2:04 PM
We also find that currently, prominent open-weight model developers often either do not implement or report on mitigations. So there is a lot of room for more innovation and information as the science grows.
In response, we cover 16 open technical problems with *unique* implications for open-weight model safety. They span the model lifecycle across training data curation, training algorithms, evaluations, deployment, and ecosystem monitoring.
November 12, 2025 at 2:04 PM
In response, we cover 16 open technical problems with *unique* implications for open-weight model safety. They span the model lifecycle across training data curation, training algorithms, evaluations, deployment, and ecosystem monitoring.
Empirical harms enabled by open models are also mounting. For example, the Internet Watch Foundation has found that they are the tools of choice for generating non-consensual AI deepfakes depicting children.
admin.iwf.org.uk/media/nadlc...
admin.iwf.org.uk/media/nadlc...
November 12, 2025 at 2:04 PM
Empirical harms enabled by open models are also mounting. For example, the Internet Watch Foundation has found that they are the tools of choice for generating non-consensual AI deepfakes depicting children.
admin.iwf.org.uk/media/nadlc...
admin.iwf.org.uk/media/nadlc...
Most importantly, powerful open-weight models are probably inevitable. For example, in recent years, they have steadily grown in their prominence, capabilities, and influence. Here are two nice graphics I often point to.
Thx @EpochAIResearch & Bhandari et al.
Thx @EpochAIResearch & Bhandari et al.
November 12, 2025 at 2:04 PM
Most importantly, powerful open-weight models are probably inevitable. For example, in recent years, they have steadily grown in their prominence, capabilities, and influence. Here are two nice graphics I often point to.
Thx @EpochAIResearch & Bhandari et al.
Thx @EpochAIResearch & Bhandari et al.
🚨New paper🚨
From a technical perspective, safeguarding open-weight model safety is AI safety in hard mode. But there's still a lot of progress to be made. Our new paper covers 16 open problems.
🧵🧵🧵
From a technical perspective, safeguarding open-weight model safety is AI safety in hard mode. But there's still a lot of progress to be made. Our new paper covers 16 open problems.
🧵🧵🧵
November 12, 2025 at 2:04 PM
🚨New paper🚨
From a technical perspective, safeguarding open-weight model safety is AI safety in hard mode. But there's still a lot of progress to be made. Our new paper covers 16 open problems.
🧵🧵🧵
From a technical perspective, safeguarding open-weight model safety is AI safety in hard mode. But there's still a lot of progress to be made. Our new paper covers 16 open problems.
🧵🧵🧵
I've essentially stopped paying attention to companies' AI eval reports. They're way too easy to game and, at this point, probably lack meaningful construct validity.
I'm increasingly persuaded that the only quantitative measures that matter anymore are usage stats & profit.
I'm increasingly persuaded that the only quantitative measures that matter anymore are usage stats & profit.
November 8, 2025 at 7:42 PM
I've essentially stopped paying attention to companies' AI eval reports. They're way too easy to game and, at this point, probably lack meaningful construct validity.
I'm increasingly persuaded that the only quantitative measures that matter anymore are usage stats & profit.
I'm increasingly persuaded that the only quantitative measures that matter anymore are usage stats & profit.
This summer, OpenAI, Anthropic, and GDM warned that their new models were nearing key risk thresholds for novice uplift on dangerous tasks.
Now that Moonshot claims Kimi K2 Thinking is SOTA, it seems, uh, less than ideal that it came with zero reporting related to safety/risk.
Now that Moonshot claims Kimi K2 Thinking is SOTA, it seems, uh, less than ideal that it came with zero reporting related to safety/risk.
November 8, 2025 at 12:22 AM
This summer, OpenAI, Anthropic, and GDM warned that their new models were nearing key risk thresholds for novice uplift on dangerous tasks.
Now that Moonshot claims Kimi K2 Thinking is SOTA, it seems, uh, less than ideal that it came with zero reporting related to safety/risk.
Now that Moonshot claims Kimi K2 Thinking is SOTA, it seems, uh, less than ideal that it came with zero reporting related to safety/risk.
Our proposal for new AI watermarking characters is officially in the Unicode document register for proposed additions. 🤞
unicode.org/L2/L2025/252...
t.co/yJfp8ezU64
unicode.org/L2/L2025/252...
t.co/yJfp8ezU64
October 21, 2025 at 2:59 PM
Our proposal for new AI watermarking characters is officially in the Unicode document register for proposed additions. 🤞
unicode.org/L2/L2025/252...
t.co/yJfp8ezU64
unicode.org/L2/L2025/252...
t.co/yJfp8ezU64
🧵🧵🧵 Do you ever hear people saying that it's important to assess AI systems based on their "marginal risk"?
Of course -- that's obvious. Nobody would ever dispute that.
So then why are we saying that?
Maybe it's a little too obvious...
Of course -- that's obvious. Nobody would ever dispute that.
So then why are we saying that?
Maybe it's a little too obvious...
October 18, 2025 at 2:00 PM
🧵🧵🧵 Do you ever hear people saying that it's important to assess AI systems based on their "marginal risk"?
Of course -- that's obvious. Nobody would ever dispute that.
So then why are we saying that?
Maybe it's a little too obvious...
Of course -- that's obvious. Nobody would ever dispute that.
So then why are we saying that?
Maybe it's a little too obvious...
🧵🧵🧵 Do you ever hear people saying that it's important to assess AI systems based on their "marginal risk"?
Of course -- that's obvious. Nobody would ever dispute that.
So then why are we saying that?
Maybe it's a little too obvious...
Of course -- that's obvious. Nobody would ever dispute that.
So then why are we saying that?
Maybe it's a little too obvious...
October 17, 2025 at 10:15 PM
🧵🧵🧵 Do you ever hear people saying that it's important to assess AI systems based on their "marginal risk"?
Of course -- that's obvious. Nobody would ever dispute that.
So then why are we saying that?
Maybe it's a little too obvious...
Of course -- that's obvious. Nobody would ever dispute that.
So then why are we saying that?
Maybe it's a little too obvious...
It draws closely from recent work that we did with @kyletokens.bsky.social et al. to mitigate risks from malicious fine-tuning.
t.co/us8MEhMrIh
t.co/us8MEhMrIh
October 9, 2025 at 10:49 PM
It draws closely from recent work that we did with @kyletokens.bsky.social et al. to mitigate risks from malicious fine-tuning.
t.co/us8MEhMrIh
t.co/us8MEhMrIh
Don't forget that in AI, "sycophancy," "pandering," "personalized alignment," "steerable alignment," and "user alignment" all describe exactly the same thing.
October 2, 2025 at 7:20 PM
Don't forget that in AI, "sycophancy," "pandering," "personalized alignment," "steerable alignment," and "user alignment" all describe exactly the same thing.