Ajitesh Shukla
@ajitesh1.bsky.social
Mathematical Research ( Geometric Topology, Differential Geometry), Large Language Models, Natural Language Processing, Quantum Computing, Cryptography, LORD KRISHNA IS GOD OF MATH
Reposted by Ajitesh Shukla
2) They find that GRPO is biased.
- The length normalization prefers shorter correct answers, and longer incorrect answers. -> length bias
- The std normalization prefers too easy or too hard questions over average questions. -> difficulty bias
They introduce Dr. GRPO to remove above biases.
- The length normalization prefers shorter correct answers, and longer incorrect answers. -> length bias
- The std normalization prefers too easy or too hard questions over average questions. -> difficulty bias
They introduce Dr. GRPO to remove above biases.
March 22, 2025 at 2:20 AM
2) They find that GRPO is biased.
- The length normalization prefers shorter correct answers, and longer incorrect answers. -> length bias
- The std normalization prefers too easy or too hard questions over average questions. -> difficulty bias
They introduce Dr. GRPO to remove above biases.
- The length normalization prefers shorter correct answers, and longer incorrect answers. -> length bias
- The std normalization prefers too easy or too hard questions over average questions. -> difficulty bias
They introduce Dr. GRPO to remove above biases.
Reposted by Ajitesh Shukla
Paper: Compute Optimal Scaling of Skills: Knowledge vs Reasoning ( arxiv.org/abs/2503.10061 )
Compute Optimal Scaling of Skills: Knowledge vs Reasoning
Scaling laws are a critical component of the LLM development pipeline, most famously as a way to forecast training decisions such as 'compute-optimally' trading-off parameter count and dataset size, a...
arxiv.org
March 21, 2025 at 8:11 PM
Paper: Compute Optimal Scaling of Skills: Knowledge vs Reasoning ( arxiv.org/abs/2503.10061 )
Reposted by Ajitesh Shukla
I think it's a limit (a slightly generalised pullback), the universal thing that completes this diagram
![A commutative diagram showing
* [0,1]^3, meaning the product of the unit interval with itself three times
* its three projections into [0,1]^2
* inclusions of the letters G, E, and B into each of those unit squares
* the universal property of the maximal subset of [0,1]^3 such that its three projections lie inside the letters G, E and B](https://cdn.bsky.app/img/feed_thumbnail/plain/did:plc:xpl57er4zbahuoaqis6qatin/bafkreigqvhevsxcmgdqgo472unhcmolhqlaaqcsd3yhtif5poqaejigtp4@jpeg)
March 20, 2025 at 5:13 PM
I think it's a limit (a slightly generalised pullback), the universal thing that completes this diagram