Here are the slides: docs.google.com/presentation...

For context the intended audience is Lang Dev researchers who are attending a session on "what can LLMs tell us about human language".

If you have any thoughts I'd love to hear them!

And it should be noted that this work might help us reimagine the nature of the computations underlying acquisition -- so for 'tokenization' isn't not entirely clear what the tokens should eventually be users.umiacs.umd.edu/~nhf/papers/...

These are fantastic, and I think there's a ton of interesting work to be done here -- because tokenization/discovery of language structure is far from trivial and definitely not 'solved' in a general sense.

I'm very excited about these models but I think we're a long way from being able to say we have in-principle solution for realistic training sizes

I think this is @glupyan.bsky.social's link: ai.meta.com/blog/textles... which includes the Zero Speech benchmarks. I agree that self-supervised models are really interesting (I'm using them in my own work right now) but as far as I know these require huge amounts of training data

My understanding @glupyan.bsky.social (correct me if I'm wrong!) is that tokenization is very much an open research area, esp. without any access to text -- I'm not aware of any BabyLLM work that examines audio-only or AV-only tokenization