_ - \. 🇺🇸
@crumb.bsky.social
https://hf.co/crumb | she / xe / it / fae / E / Ey
I'm so excited to see other people discovering this solution!
I've been doing literally this but for general completions. this lets a model learn to reason about _everything present in the web text corpus_
I've been doing literally this but for general completions. this lets a model learn to reason about _everything present in the web text corpus_
December 20, 2025 at 6:06 PM
I'm so excited to see other people discovering this solution!
I've been doing literally this but for general completions. this lets a model learn to reason about _everything present in the web text corpus_
I've been doing literally this but for general completions. this lets a model learn to reason about _everything present in the web text corpus_
we did some scraping and data engineering and have 2 billion tokens to use to model posts that users reply to and their replies to them given their recent activity
December 16, 2025 at 6:42 AM
we did some scraping and data engineering and have 2 billion tokens to use to model posts that users reply to and their replies to them given their recent activity
GAN approach w/ reasoning seems like it's gonna be stable at small batch sizes - 4B @ bs 8
December 8, 2025 at 12:56 AM
GAN approach w/ reasoning seems like it's gonna be stable at small batch sizes - 4B @ bs 8
actually you dont even need to do Both things there in some cases, this is one of my favorite applications of RL
arxiv.org/abs/1912.04871
arxiv.org/abs/1912.04871
December 8, 2025 at 12:55 AM
actually you dont even need to do Both things there in some cases, this is one of my favorite applications of RL
arxiv.org/abs/1912.04871
arxiv.org/abs/1912.04871
that looked so messy as a human so i am trying out some different reskins, "computers think in binary"?
November 30, 2025 at 12:43 AM
that looked so messy as a human so i am trying out some different reskins, "computers think in binary"?
are you guys chill with this or do i have safety geeks that will crucify me over here
November 28, 2025 at 8:17 PM
are you guys chill with this or do i have safety geeks that will crucify me over here
have been revisiting this a lot
youtu.be/0BVM0UC28nY
youtu.be/0BVM0UC28nY
September 30, 2025 at 1:43 AM
have been revisiting this a lot
youtu.be/0BVM0UC28nY
youtu.be/0BVM0UC28nY
even tho we trained on filtered data generated by deepseek v3 base, our desc2doc model didn't follow prompts as well as we'd hoped. so last night i pounded out a rubric based trainer using deepseek v3.1 (:free) as judge. it is now running. yaaay
September 29, 2025 at 7:05 PM
even tho we trained on filtered data generated by deepseek v3 base, our desc2doc model didn't follow prompts as well as we'd hoped. so last night i pounded out a rubric based trainer using deepseek v3.1 (:free) as judge. it is now running. yaaay
took you long enough Dumb Ass
September 18, 2025 at 3:29 AM
took you long enough Dumb Ass
lets go man fuck em up 𝔱𝔬𝔲𝔤𝔥-𝔡𝔯𝔞𝔤𝔬𝔫-₂₅₈
ETA83:50:08
ETA83:50:08
September 3, 2025 at 11:33 PM
lets go man fuck em up 𝔱𝔬𝔲𝔤𝔥-𝔡𝔯𝔞𝔤𝔬𝔫-₂₅₈
ETA83:50:08
ETA83:50:08
subtracting "lamb" embed from mary had a little lamb embed then decoding... it tries to say it but it just cant get it right... that's so silly...
September 2, 2025 at 5:17 AM
subtracting "lamb" embed from mary had a little lamb embed then decoding... it tries to say it but it just cant get it right... that's so silly...
trying strange things
September 2, 2025 at 4:54 AM
trying strange things
okokokok it's on HF as it is RN, it seems really good but it will keep on improving for a little while,
encourage you to try it out and see if you can figure out any fun things to use it for
hf.co/crumb/essenc...
encourage you to try it out and see if you can figure out any fun things to use it for
hf.co/crumb/essenc...
September 2, 2025 at 4:40 AM
okokokok it's on HF as it is RN, it seems really good but it will keep on improving for a little while,
encourage you to try it out and see if you can figure out any fun things to use it for
hf.co/crumb/essenc...
encourage you to try it out and see if you can figure out any fun things to use it for
hf.co/crumb/essenc...
we want to do 8b but that requires offloading to CPU in our case which is just... not gonna cut it when the training time is going to start being in the 10ks of steps
August 28, 2025 at 6:35 PM
we want to do 8b but that requires offloading to CPU in our case which is just... not gonna cut it when the training time is going to start being in the 10ks of steps
it took a bit of tinkering crumb had posted this on 🐦 2days ago
August 28, 2025 at 6:35 PM
it took a bit of tinkering crumb had posted this on 🐦 2days ago
this one is for the freaks, have u ever wanted a text2vec2text that 1 doesn't rely on api embeddings and 2 preserves temporal dynamics by design?
crumb has found crumbself in a position in need of some of these, so crumb is jst building them. 32 token embedding. total 6b model system (WIP results)
crumb has found crumbself in a position in need of some of these, so crumb is jst building them. 32 token embedding. total 6b model system (WIP results)
August 28, 2025 at 6:33 PM
this one is for the freaks, have u ever wanted a text2vec2text that 1 doesn't rely on api embeddings and 2 preserves temporal dynamics by design?
crumb has found crumbself in a position in need of some of these, so crumb is jst building them. 32 token embedding. total 6b model system (WIP results)
crumb has found crumbself in a position in need of some of these, so crumb is jst building them. 32 token embedding. total 6b model system (WIP results)
and cogview.. remember cogview
August 25, 2025 at 3:36 PM
and cogview.. remember cogview
and the beginning of an RNN crumb was training for Q/A around same time, rage quit after large N runs or class period ended and had to close chrome book LOL
August 25, 2025 at 3:36 PM
and the beginning of an RNN crumb was training for Q/A around same time, rage quit after large N runs or class period ended and had to close chrome book LOL
crumb found a trove of stuff crumb generated in 2019
August 25, 2025 at 3:36 PM
crumb found a trove of stuff crumb generated in 2019
why does huggingface have no thumbs down react
August 19, 2025 at 5:52 PM
why does huggingface have no thumbs down react
crumb got it working on qwen 2.5 32b for
- llm response
- llm prompt
- llm conversation
- samples from dclm
- samples from textfiles dot com
needs a little tuning and then can be specialized into many many things (again, crumb excited for rl on "llm prompt")
- llm response
- llm prompt
- llm conversation
- samples from dclm
- samples from textfiles dot com
needs a little tuning and then can be specialized into many many things (again, crumb excited for rl on "llm prompt")
August 18, 2025 at 6:10 AM
crumb got it working on qwen 2.5 32b for
- llm response
- llm prompt
- llm conversation
- samples from dclm
- samples from textfiles dot com
needs a little tuning and then can be specialized into many many things (again, crumb excited for rl on "llm prompt")
- llm response
- llm prompt
- llm conversation
- samples from dclm
- samples from textfiles dot com
needs a little tuning and then can be specialized into many many things (again, crumb excited for rl on "llm prompt")