pranav
@pranav.bsky.social
Research Scientist at Google DeepMind. ಕನ್ನಡಿಗ.
Past: Researchoor, Algorithms team at OpenAI & with Juergen Schmidhuber.
Past: Researchoor, Algorithms team at OpenAI & with Juergen Schmidhuber.
Everything about that Luigi guy is just sad. Manifesto written by an LSTM, no big plan or idea, got caught in the dumbest way imaginable, miserable health condition.. just sad overall
December 11, 2024 at 1:56 AM
Everything about that Luigi guy is just sad. Manifesto written by an LSTM, no big plan or idea, got caught in the dumbest way imaginable, miserable health condition.. just sad overall
mom pick me up they’re putting air quotes on integrals now
December 8, 2024 at 11:18 PM
mom pick me up they’re putting air quotes on integrals now
Commits should be first class objects. VCS is not some outer loop feature. It is what we do
December 3, 2024 at 8:20 PM
Commits should be first class objects. VCS is not some outer loop feature. It is what we do
grateful for unga buga loss go down technology
🦃🇺🇸💸
🦃🇺🇸💸
November 28, 2024 at 3:57 PM
grateful for unga buga loss go down technology
🦃🇺🇸💸
🦃🇺🇸💸
Importance sampling does not work in deep learning
This suggests that SGD is a frequentist process, not a bayesian one
proceedings.mlr.press/v97/byrd19a/...
This suggests that SGD is a frequentist process, not a bayesian one
proceedings.mlr.press/v97/byrd19a/...
proceedings.mlr.press
November 26, 2024 at 8:32 PM
Importance sampling does not work in deep learning
This suggests that SGD is a frequentist process, not a bayesian one
proceedings.mlr.press/v97/byrd19a/...
This suggests that SGD is a frequentist process, not a bayesian one
proceedings.mlr.press/v97/byrd19a/...
Everybody gangsta until deepseek r1 starts thinking in Chinese
November 26, 2024 at 6:47 PM
Everybody gangsta until deepseek r1 starts thinking in Chinese
Looks like Sutton didn’t get the memo. Memory and compute are cheap. It’s called the “Bitter Lesson”
Replay buffers are a-ok!
Replay buffers are a-ok!
Soooo....... Rich Sutton's team just leapfrogged the whole field with this banger, solving RL with pure streaming data:
arxiv.org/abs/2410.14606
arxiv.org/abs/2410.14606
November 26, 2024 at 2:33 AM
Looks like Sutton didn’t get the memo. Memory and compute are cheap. It’s called the “Bitter Lesson”
Replay buffers are a-ok!
Replay buffers are a-ok!
Eric Schmidt has written a second book with Henry Kissinger on AI. Incredible
November 25, 2024 at 2:57 PM
Eric Schmidt has written a second book with Henry Kissinger on AI. Incredible
Reposted by pranav
distributed learning for LLM?
recently, @primeintellect.bsky.social have announced finishing their 10B distributed learning, trained across the world.
what is it exactly?
🧵
recently, @primeintellect.bsky.social have announced finishing their 10B distributed learning, trained across the world.
what is it exactly?
🧵
November 25, 2024 at 12:02 PM
distributed learning for LLM?
recently, @primeintellect.bsky.social have announced finishing their 10B distributed learning, trained across the world.
what is it exactly?
🧵
recently, @primeintellect.bsky.social have announced finishing their 10B distributed learning, trained across the world.
what is it exactly?
🧵
Death of pre-training has been greatly exaggerated.
Look at image generation models. We’re so far from compute optimality yet pre-training is by far the most important bit.
Diffusion models are like o1: train for 1 step but unroll for many. Even test time inference is pre-training bottlenecked.
Look at image generation models. We’re so far from compute optimality yet pre-training is by far the most important bit.
Diffusion models are like o1: train for 1 step but unroll for many. Even test time inference is pre-training bottlenecked.
November 23, 2024 at 5:51 PM
Death of pre-training has been greatly exaggerated.
Look at image generation models. We’re so far from compute optimality yet pre-training is by far the most important bit.
Diffusion models are like o1: train for 1 step but unroll for many. Even test time inference is pre-training bottlenecked.
Look at image generation models. We’re so far from compute optimality yet pre-training is by far the most important bit.
Diffusion models are like o1: train for 1 step but unroll for many. Even test time inference is pre-training bottlenecked.
Stop sharing lists and start posting content
November 21, 2024 at 8:27 AM
Stop sharing lists and start posting content
Reposted by pranav
Gonna start posting I’m posting again we’re so back
November 20, 2024 at 4:19 AM
Gonna start posting I’m posting again we’re so back
People think scaling laws started with Kaplan et al.
But almost 3 years before that this paper came out with power law fits, data and compute scaling laws on language modeling.
arxiv.org/abs/1712.00409
Goes to show how having the right idea is only a small part of success
But almost 3 years before that this paper came out with power law fits, data and compute scaling laws on language modeling.
arxiv.org/abs/1712.00409
Goes to show how having the right idea is only a small part of success
Deep Learning Scaling is Predictable, Empirically
Deep learning (DL) creates impactful advances following a virtuous recipe: model architecture search, creating large training data sets, and scaling computation. It is widely believed that growing tra...
arxiv.org
November 20, 2024 at 4:43 AM
People think scaling laws started with Kaplan et al.
But almost 3 years before that this paper came out with power law fits, data and compute scaling laws on language modeling.
arxiv.org/abs/1712.00409
Goes to show how having the right idea is only a small part of success
But almost 3 years before that this paper came out with power law fits, data and compute scaling laws on language modeling.
arxiv.org/abs/1712.00409
Goes to show how having the right idea is only a small part of success
Twitter <> ChatGPT
BlueSky <> Claude
Threads <> Gemini
BlueSky <> Claude
Threads <> Gemini
November 19, 2024 at 9:18 AM
Twitter <> ChatGPT
BlueSky <> Claude
Threads <> Gemini
BlueSky <> Claude
Threads <> Gemini
All right round 2 of bluesky. Hope this is it.
November 15, 2024 at 11:49 PM
All right round 2 of bluesky. Hope this is it.
Reposted by pranav
hard for me to get engagement on this platform but it’s ok I’m just waiting for all the AI researchers to show up so I can go mega viral saying shit like “who called it Xavier Initialization and not Weighting For Glorot”
July 2, 2023 at 5:27 PM
hard for me to get engagement on this platform but it’s ok I’m just waiting for all the AI researchers to show up so I can go mega viral saying shit like “who called it Xavier Initialization and not Weighting For Glorot”
wen reverse scandal finds you
Haven’t seen much news about this app lately.
Guyz we need a scandal
Guyz we need a scandal
July 1, 2023 at 9:04 PM
wen reverse scandal finds you
If we’re taking shoes off forever did Osama really lose?
June 26, 2023 at 12:48 PM
If we’re taking shoes off forever did Osama really lose?
Haven’t seen much news about this app lately.
Guyz we need a scandal
Guyz we need a scandal
June 24, 2023 at 5:04 AM
Haven’t seen much news about this app lately.
Guyz we need a scandal
Guyz we need a scandal
Going on an impromptu hike
Rated challenging
In an unknown country
Zero sleep
Not prepared re clothes/hat/shoes or food/water
Need to come back and pack for an early morning flight immediately after
Rain, even thunderstorms forecast on the trail
Let’s see what I make of this in some time
Rated challenging
In an unknown country
Zero sleep
Not prepared re clothes/hat/shoes or food/water
Need to come back and pack for an early morning flight immediately after
Rain, even thunderstorms forecast on the trail
Let’s see what I make of this in some time
June 18, 2023 at 5:30 AM
Going on an impromptu hike
Rated challenging
In an unknown country
Zero sleep
Not prepared re clothes/hat/shoes or food/water
Need to come back and pack for an early morning flight immediately after
Rain, even thunderstorms forecast on the trail
Let’s see what I make of this in some time
Rated challenging
In an unknown country
Zero sleep
Not prepared re clothes/hat/shoes or food/water
Need to come back and pack for an early morning flight immediately after
Rain, even thunderstorms forecast on the trail
Let’s see what I make of this in some time
Reposted by pranav
“I asked an AI program to write an entire sci-fi novel for me, and [as someone who will die an empty shell of a man who wasted his life doing nothing for the world...] I was super impressed.”
https://www.theonion.com/guy-who-sucks-at-being-a-person-sees-huge-potential-in-1850488022
😅
https://www.theonion.com/guy-who-sucks-at-being-a-person-sees-huge-potential-in-1850488022
😅
Guy Who Sucks At Being A Person Sees Huge Potential In AI
SAN MATEO, CA—After spending the past three decades of his life being totally unable and unwilling to engage in any meaningful way with the world around him, James Parker, a local guy who sucks at b...
www.theonion.com
June 12, 2023 at 5:23 PM
“I asked an AI program to write an entire sci-fi novel for me, and [as someone who will die an empty shell of a man who wasted his life doing nothing for the world...] I was super impressed.”
https://www.theonion.com/guy-who-sucks-at-being-a-person-sees-huge-potential-in-1850488022
😅
https://www.theonion.com/guy-who-sucks-at-being-a-person-sees-huge-potential-in-1850488022
😅
I’ve always been mixed about Sam Altman. It’s kinda confusing because I like most of the things he says and does.
Then again he says things like this and my confusion is clear to me:
https://twitter.com/ETNOWlive/status/1666460799093620738?s=20
Then again he says things like this and my confusion is clear to me:
https://twitter.com/ETNOWlive/status/1666460799093620738?s=20
June 8, 2023 at 6:57 PM
I’ve always been mixed about Sam Altman. It’s kinda confusing because I like most of the things he says and does.
Then again he says things like this and my confusion is clear to me:
https://twitter.com/ETNOWlive/status/1666460799093620738?s=20
Then again he says things like this and my confusion is clear to me:
https://twitter.com/ETNOWlive/status/1666460799093620738?s=20
why did i pick up the phone and what am i doing
June 5, 2023 at 2:22 AM
why did i pick up the phone and what am i doing