Bill Dally presenting the rise in necessary compute for AI

I see why this is good for Nvidia. But why would this be considered good for anyone else?

At the @cra-hq.bsky.social CCC Computing Futures Symposium this week - exciting (and interesting) times to be talking about the future of computing research!

As part of this event, last night we demo'd neuromorphic and its potential impact on ModSim and #NeuroAI at the US Senate!

If the endless "some people optimized LLMs" news is starting to bore you, check out our preprint on neuromorphic-compatible neural circuits that can solve sparse linear systems.

arxiv.org/abs/2501.10526

This opens up some exciting new doors for brain-like algorithms! 🧪🧠🤖

#NeuroAI #Neuromorphic

I'm not sure about the connection to SGD in large ML models. What I have always seen boils down to this figure. Naively, as scale (# inputs) goes up, a constant E/I ratio of inputs to a neuron the expected value of firing stays the same but the sensitivity of the firing rate to any variability grows

My new office art

This is all sorts of fun.

2) More interestingly, the 2nd most popular choice reflects uncertainty you would naively expect.  Shirts and jackets on Fashion MNIST; dogs and deer on CIFAR-10.  1’s and 7’s on MNIST. 3) This amazingly seems pretty robust to precision in the probabilities. 7/

The paper – which is pretty short – goes into all of the details. But punchlines: 1) it works. With minimal impact on training, majority vote of samples gets pretty close to deterministic accuracy in classification. 6/