The bug was in OpenReview's core, not in the conference setup, as I understand it bsky.app/profile/cspr...

SFI Visual Intelligence is here as @sfi-vi.bsky.social :)

I am sorry to hear that, it's a great paper and having shared permissively licensed and (even though SYNTH is great) non-synthetic pretraining set is really important for more rigorous discussion of LM design choices, IMO

If you haven't seen it, you might also like Lyttle Lytton, the ongoing short-form contest inspired by it: adamcadre.ac/lyttle/

Boring: Releasing models via Twitter

Exciting: Releasing models via twitter

Add overcomplete autoencoders before the output layer until your model is >1B, problem solved ;)

More seriously, is this a criticism you've gotten or is it coming up in more reviews? Frustrating either way but I really hope nobody's using "more params = bigger lab = better than" as a rule of thumb

(The SuperDisk was a diskette drive which had its own 120 MB disks but could also be used to format 3.5" diskettes to store 32 MB, which one person used to make a diskette version of Factorio: www.reddit.com/r/factorio/c...)

At 56M parameters, naive quantization *may* not be the best way to go

Do they have the consensus play into the training in any way? I don't want to assume it's just "models are vulnerable to adversarial input samples, so we will only have signed input samples, which may still be adversarial but now with a public key"

There is at least one project which implements the GPT-2 forward pass in WebGL, using fragment shaders for the different layers, which does let you visualize the intermediate steps: github.com/nathan-barry...