I don't know about you but the way my brain works is by analyzing the contents of the entire internet to make an educated guess about what word I should use next.

Are we really at a stage in public education where we consider it OK to have literally Google-branded schoolchildren whose learner identities are tied to being "responsible AI" users of private for-profit technologies?

Correct. No notes

And “patriarch maxxing” was right there…

Now, we should think about what the questions are & how we can answer them.

An important question is: how is the brain capable of bootstrapping structure from statistics? And the reverse: does the brain refine probabilistic representations with structured knowledge, and if so, how does this work?

This means that any effects found for surprisal always leave room for the possibility of latent factors driving both the probabilities and the human responses, and do not allow any conclusions about which factors are involved (and why).

So... Now what?

(image by Noémie te Rietmolen)

By contrast, using a data-driven feature like surprisal as an explanation prevents us from looking at the influence from latent factors by reflecting variance that stems from these factors as a second-order variable.

The problem with this power is that data-driven estimate will perform better than a theory-driven estimate. Because the data do not err, the theorizer does (@olivia.science & @andreaeyleen.bsky.social, 2021). These mistakes are awesome: they are opportunities to adjust our theory!

The power of surprisal stems from the fact that (lexical) surprisal can —and will— parametrically reflect variation stemming from any domain or representational level of language. Why? Because words form patterns for many reasons! Semantics, syntax, frequency... Surprisal does not distinguish.

Surprisal is the ‘everything bagel/nothing burger’ of predictors—it has everything baked in, which is the problem.