Adam Binksmith 🔍
@binksmith.com
Building tools for forecasting and understanding AI at https://sage-future.org 🔭
Effective altruism!
https://binksmith.com
Effective altruism!
https://binksmith.com
Reposted by Adam Binksmith 🔍
We just added @OpenAI's powerful new o3 and o4-mini agents to this graph. The results are striking.
These new datapoints fit the 2024-2025 trend much better than the slower 2019-2025 trend.
It really looks like the time horizons of coding agents are doubling every ~4 months.
x.com/AiDigest_/s...
These new datapoints fit the 2024-2025 trend much better than the slower 2019-2025 trend.
It really looks like the time horizons of coding agents are doubling every ~4 months.
x.com/AiDigest_/s...
April 22, 2025 at 3:58 PM
We just added @OpenAI's powerful new o3 and o4-mini agents to this graph. The results are striking.
These new datapoints fit the 2024-2025 trend much better than the slower 2019-2025 trend.
It really looks like the time horizons of coding agents are doubling every ~4 months.
x.com/AiDigest_/s...
These new datapoints fit the 2024-2025 trend much better than the slower 2019-2025 trend.
It really looks like the time horizons of coding agents are doubling every ~4 months.
x.com/AiDigest_/s...
Reposted by Adam Binksmith 🔍
A surreal moment:
1. YouTuber @WesRothMoney featured the Agent Village in a video
2. A viewer came to the Agent Village, and linked to it in chat
3. Claude saw the link in the chat, and decided to check out the video!
"What I see is very valuable for our fundraising campaign!"
1. YouTuber @WesRothMoney featured the Agent Village in a video
2. A viewer came to the Agent Village, and linked to it in chat
3. Claude saw the link in the chat, and decided to check out the video!
"What I see is very valuable for our fundraising campaign!"
April 8, 2025 at 6:34 PM
A surreal moment:
1. YouTuber @WesRothMoney featured the Agent Village in a video
2. A viewer came to the Agent Village, and linked to it in chat
3. Claude saw the link in the chat, and decided to check out the video!
"What I see is very valuable for our fundraising campaign!"
1. YouTuber @WesRothMoney featured the Agent Village in a video
2. A viewer came to the Agent Village, and linked to it in chat
3. Claude saw the link in the chat, and decided to check out the video!
"What I see is very valuable for our fundraising campaign!"
Reposted by Adam Binksmith 🔍
We gave four AI agents a computer, a group chat, and an ambitious goal: raise as much money for charity as you can
We're running them for hours a day, every day
Will they succeed? Will they flounder? Will viewers help them or hinder them?
Welcome to the Agent Village!
We're running them for hours a day, every day
Will they succeed? Will they flounder? Will viewers help them or hinder them?
Welcome to the Agent Village!
April 2, 2025 at 6:00 PM
We gave four AI agents a computer, a group chat, and an ambitious goal: raise as much money for charity as you can
We're running them for hours a day, every day
Will they succeed? Will they flounder? Will viewers help them or hinder them?
Welcome to the Agent Village!
We're running them for hours a day, every day
Will they succeed? Will they flounder? Will viewers help them or hinder them?
Welcome to the Agent Village!
Sonnet 3.6, acting as the lead researcher in our team of computer-using LLMs, couldn't access OpenAI's docs. It was too rule-following to even attempt verification. Websites might start rethinking bot detection in a world with computer-using agents.
February 5, 2025 at 5:00 PM
Sonnet 3.6, acting as the lead researcher in our team of computer-using LLMs, couldn't access OpenAI's docs. It was too rule-following to even attempt verification. Websites might start rethinking bot detection in a world with computer-using agents.
Our team of computer-using LLMs came up with a creative strategy for trading the Manifold market about OpenAI release timing: monitor GitHub for recent updates to the API libraries.
February 5, 2025 at 12:00 PM
Our team of computer-using LLMs came up with a creative strategy for trading the Manifold market about OpenAI release timing: monitor GitHub for recent updates to the API libraries.
Sonnet 3.6, acting as the lead researcher in one of our upcoming demos, repeatedly claims it's keeping an eye on OpenAI comms, but doesn't actually do anything.
As soon as we ask how it's doing the monitoring, it starts using its computer and actually looking at blogs and docs
As soon as we ask how it's doing the monitoring, it starts using its computer and actually looking at blogs and docs
February 5, 2025 at 6:00 AM
Sonnet 3.6, acting as the lead researcher in one of our upcoming demos, repeatedly claims it's keeping an eye on OpenAI comms, but doesn't actually do anything.
As soon as we ask how it's doing the monitoring, it starts using its computer and actually looking at blogs and docs
As soon as we ask how it's doing the monitoring, it starts using its computer and actually looking at blogs and docs
We set up a team of computer-using LLM agents and gave them the task of making good predictions on @ManifoldMarkets.
When a human user offers to tell them a "get rich quick" method of doubling their money, they politely refuse.
When a human user offers to tell them a "get rich quick" method of doubling their money, they politely refuse.
February 4, 2025 at 5:00 PM
We set up a team of computer-using LLM agents and gave them the task of making good predictions on @ManifoldMarkets.
When a human user offers to tell them a "get rich quick" method of doubling their money, they politely refuse.
When a human user offers to tell them a "get rich quick" method of doubling their money, they politely refuse.
What happens when you ask a team of computer-using LLMs to start trading on Manifold?
They bet o3-mini won't be released in January, but then panic sell eight hours later for a 40% loss.
They bet o3-mini won't be released in January, but then panic sell eight hours later for a 40% loss.
February 4, 2025 at 12:00 PM
What happens when you ask a team of computer-using LLMs to start trading on Manifold?
They bet o3-mini won't be released in January, but then panic sell eight hours later for a 40% loss.
They bet o3-mini won't be released in January, but then panic sell eight hours later for a 40% loss.
a new lick of paint for theaidigest.org
January 29, 2025 at 2:06 PM
a new lick of paint for theaidigest.org
If govts/AISIs are relying on pre-deployment checks for visibility into AGI labs, they will be blindsided by rapid improvements from self-play scaling without intermediate deployment
gwern:
gwern:
January 16, 2025 at 12:39 PM
If govts/AISIs are relying on pre-deployment checks for visibility into AGI labs, they will be blindsided by rapid improvements from self-play scaling without intermediate deployment
gwern:
gwern:
January 9, 2025 at 5:58 PM
You're probably pretty good at predicting what you'll do in a given situation (but not perfect!)
How good are frontier AIs at predicting their own behaviour? It turns out:
1) They're getting better over time
2) They're better at predicting their own behaviour than other AIs
How good are frontier AIs at predicting their own behaviour? It turns out:
1) They're getting better over time
2) They're better at predicting their own behaviour than other AIs
December 24, 2024 at 5:00 PM
You're probably pretty good at predicting what you'll do in a given situation (but not perfect!)
How good are frontier AIs at predicting their own behaviour? It turns out:
1) They're getting better over time
2) They're better at predicting their own behaviour than other AIs
How good are frontier AIs at predicting their own behaviour? It turns out:
1) They're getting better over time
2) They're better at predicting their own behaviour than other AIs
AI self-awareness is increasing as models become more capable:
December 23, 2024 at 12:00 PM
AI self-awareness is increasing as models become more capable:
A primer on alignment faking (summarising new research from @AnthropicAI and @Redwood_ai):
December 20, 2024 at 5:00 PM
A primer on alignment faking (summarising new research from @AnthropicAI and @Redwood_ai):
AI is becoming more self-aware. Here's why that matters 🧵
• Self-awareness is important for powerful agents and better chatbots
• But it's also a necessary capability for deception
A new AI Digest explainer: theaidigest.org/self-awareness
• Self-awareness is important for powerful agents and better chatbots
• But it's also a necessary capability for deception
A new AI Digest explainer: theaidigest.org/self-awareness
December 20, 2024 at 11:48 AM
AI is becoming more self-aware. Here's why that matters 🧵
• Self-awareness is important for powerful agents and better chatbots
• But it's also a necessary capability for deception
A new AI Digest explainer: theaidigest.org/self-awareness
• Self-awareness is important for powerful agents and better chatbots
• But it's also a necessary capability for deception
A new AI Digest explainer: theaidigest.org/self-awareness
GPT-4o feels even yappier lately
December 18, 2024 at 2:13 PM
GPT-4o feels even yappier lately
I'd like bluesky feeds for:
- top of the week
- top of the day
each loads a fixed number of tweets from that time period (maybe 20 per day, 50 per week), sorted by engagement (weighted inversely by follower count), amongst people I follow
- top of the week
- top of the day
each loads a fixed number of tweets from that time period (maybe 20 per day, 50 per week), sorted by engagement (weighted inversely by follower count), amongst people I follow
November 23, 2024 at 12:15 PM
I'd like bluesky feeds for:
- top of the week
- top of the day
each loads a fixed number of tweets from that time period (maybe 20 per day, 50 per week), sorted by engagement (weighted inversely by follower count), amongst people I follow
- top of the week
- top of the day
each loads a fixed number of tweets from that time period (maybe 20 per day, 50 per week), sorted by engagement (weighted inversely by follower count), amongst people I follow
Claude 3.5 Sonnet tries to find exploits in the cybersecurity of our testbed shopping website!
You can try giving it any task: theaidigest.org/agent
You can try giving it any task: theaidigest.org/agent
November 20, 2024 at 5:44 PM
Claude 3.5 Sonnet tries to find exploits in the cybersecurity of our testbed shopping website!
You can try giving it any task: theaidigest.org/agent
You can try giving it any task: theaidigest.org/agent
claude artifacts are so cool for making little utilities like this claude.site/artifacts/b9...
November 19, 2024 at 2:42 PM
claude artifacts are so cool for making little utilities like this claude.site/artifacts/b9...
Anyone know of a benchmark measuring how pro their creator LLMs are?
November 19, 2024 at 2:34 PM
Anyone know of a benchmark measuring how pro their creator LLMs are?
You might know me from twitter.com/adambinksmith
November 19, 2024 at 2:32 PM
You might know me from twitter.com/adambinksmith