What ideas are already out there, just waiting on someone to really feel their power and bring them down from the ivory tower?

During questions someone asked what we can learn about how to write an influential paper. Equally important is what we can learn about reading such a paper. So many philosophers had read it in the intervening generation, but none had taken it seriously.

It made me realise for the first time that I was essential in making it so — that one Australian in Oxford in 1971 had thrown the ball far far down the field, to be received by another Australian in Oxford in 2004.

If you want to go a little deeper, see my full post:
www.tobyord.com/writing/most...
14/14

So it looks like most of the gains are coming from the ability to spend more compute on each answer rather than from better ability to reason for the same token budget.
This shift to inference-scaling has big implications for AI business, governance, and risk:
www.tobyord.com/writing/infe...
13/

And here are the relative boosts.
Overall the inference scaling produced 82%, 63%, and 92% of the total performance gains on the different benchmarks.
12/

As you can see, most of the boost is coming from the inference-scaling that the RL training has enabled.
The same is true for the other benchmarks I examined. Here are the raw scatterplots:
11/

We can draw the trend on the chart, then divide the performance boost in two:
• the RL boost taking the base model to the trend line
• the inference-scaling boost taking it to the top of the trend
10/

Note how there is a clear trend line for the reasoning models, showing how their performance scales with more inference. The base model is slightly below this trend.
9/

I worked out a nice clean way to separate this out. Here is data from the MATH level 5 benchmark, showing performance vs token-use for a base model (Sonnet 3.6 – orange square) and its reasoning model (Sonnet 3.7 – red circles).
8/

But it turns out that even when reasoning is turned off, these models are using many more tokens to generate their answers, so even this boost is partly just from RL and partly from the inference-scaling.
7/

Often people assume it is mostly about the training. One piece of evidence for this is that even without reasoning turned on, a reasoning model seems to perform substantially better than its base model (i.e. a model that differs only in not having the RL training)
6/

But it is hard to tease out how much of the benefits of RL are coming directly from the training (1) and how much are coming from using far more tokens to run it (2).
5/

But (2) is less rosy.
For the largest AI companies, most costs come from deploying models to customers. If you need to 10x or 100x those costs, that is very expensive. And unlike training, it can't be made up in volume.
4/

Many people focus on (1).
This is the bull case for RL scaling — it started off small compared to internet-scale pre-training, so can be scaled 10x or 100x before doubling overall training compute.
3/

Scaling up AI using next-token prediction was the most important trend in modern AI. It stalled out over the last couple of years and has been replaced by RL scaling.

This has two parts:
1. Scaling RL training
2. Scaling inference compute at deployment

2/