@jebediah98.bsky.social
Oh ok I didn’t know that!
November 18, 2025 at 5:33 PM
Oh ok I didn’t know that!
Do you think this is mainly because some businesses are strategically locked into Azure? Claude seems to mainly be targeting business/API.
November 18, 2025 at 4:48 PM
Do you think this is mainly because some businesses are strategically locked into Azure? Claude seems to mainly be targeting business/API.
He was correct way ahead of time on other things such as spin networks being integral to quantum gravity.
Considering anaesthesia and possible link to quantum decoherence is well, very clever at least.
I doubt it is true myself, fwiw.
Considering anaesthesia and possible link to quantum decoherence is well, very clever at least.
I doubt it is true myself, fwiw.
November 18, 2025 at 4:43 PM
He was correct way ahead of time on other things such as spin networks being integral to quantum gravity.
Considering anaesthesia and possible link to quantum decoherence is well, very clever at least.
I doubt it is true myself, fwiw.
Considering anaesthesia and possible link to quantum decoherence is well, very clever at least.
I doubt it is true myself, fwiw.
It’s an over used term, but I think Penrose is one of few geniuses alive today.
He has argued very well and defended his position on the quantum brain for decades (also some experiments support the idea). I don’t think it’s reasonable to dismiss the possibility that he is correct.
He has argued very well and defended his position on the quantum brain for decades (also some experiments support the idea). I don’t think it’s reasonable to dismiss the possibility that he is correct.
November 18, 2025 at 4:36 PM
It’s an over used term, but I think Penrose is one of few geniuses alive today.
He has argued very well and defended his position on the quantum brain for decades (also some experiments support the idea). I don’t think it’s reasonable to dismiss the possibility that he is correct.
He has argued very well and defended his position on the quantum brain for decades (also some experiments support the idea). I don’t think it’s reasonable to dismiss the possibility that he is correct.
The noncomputable/quantum thing is just a question of science. Penrose has proposed this and stood by that position for decades. If it turns out to be true, then it would mean that none of our current computers could ever have the exact same capabilities as a human.
Most would call it magic or soul
Most would call it magic or soul
November 18, 2025 at 10:18 AM
The noncomputable/quantum thing is just a question of science. Penrose has proposed this and stood by that position for decades. If it turns out to be true, then it would mean that none of our current computers could ever have the exact same capabilities as a human.
Most would call it magic or soul
Most would call it magic or soul
For (certain subfields of) maths and some others old books (out of copyright) can still be good. For science have to be careful because it’s changed a lot last 100 years. Would need alternative to text book for that, I agree.
Also btw the name of the model is truly legendary.
Also btw the name of the model is truly legendary.
November 11, 2025 at 11:46 AM
For (certain subfields of) maths and some others old books (out of copyright) can still be good. For science have to be careful because it’s changed a lot last 100 years. Would need alternative to text book for that, I agree.
Also btw the name of the model is truly legendary.
Also btw the name of the model is truly legendary.
The name is simply unmatched. I think their methodology probably scales quite high too. Possibly even to 120b MoE size, ie a way to get frontier with data that can be released.
November 11, 2025 at 11:28 AM
The name is simply unmatched. I think their methodology probably scales quite high too. Possibly even to 120b MoE size, ie a way to get frontier with data that can be released.
I was intuitively thinking Wikipedia -> Selected Textbooks?
200b -> 5T tokens (same as current fully open sota like Marin/AllenAI pretrain).
Your method can also scale far further because it provides a methodology for compliant licensing.
200b -> 5T tokens (same as current fully open sota like Marin/AllenAI pretrain).
Your method can also scale far further because it provides a methodology for compliant licensing.
November 11, 2025 at 11:16 AM
I was intuitively thinking Wikipedia -> Selected Textbooks?
200b -> 5T tokens (same as current fully open sota like Marin/AllenAI pretrain).
Your method can also scale far further because it provides a methodology for compliant licensing.
200b -> 5T tokens (same as current fully open sota like Marin/AllenAI pretrain).
Your method can also scale far further because it provides a methodology for compliant licensing.
Amazing work! This looks like the secret to making smaller models.
I wonder if this would scale to making a larger model like gpt oss 120b in capability. Would need more tokens but assuming similar scaling far less than 2.1 million H100 hours worth of tokens.
I wonder if this would scale to making a larger model like gpt oss 120b in capability. Would need more tokens but assuming similar scaling far less than 2.1 million H100 hours worth of tokens.
November 11, 2025 at 10:00 AM
Amazing work! This looks like the secret to making smaller models.
I wonder if this would scale to making a larger model like gpt oss 120b in capability. Would need more tokens but assuming similar scaling far less than 2.1 million H100 hours worth of tokens.
I wonder if this would scale to making a larger model like gpt oss 120b in capability. Would need more tokens but assuming similar scaling far less than 2.1 million H100 hours worth of tokens.
Monopolisation was and still is my greatest fear. I agree that the risk is looking reduced now compared to a year ago though.
November 9, 2025 at 2:56 AM
Monopolisation was and still is my greatest fear. I agree that the risk is looking reduced now compared to a year ago though.
Ok yeah this got me!
October 31, 2025 at 10:43 PM
Ok yeah this got me!
Would be interesting to see Qwen max/coder, DeepSeek 3.2 and Minimax.
October 31, 2025 at 2:08 AM
Would be interesting to see Qwen max/coder, DeepSeek 3.2 and Minimax.
It’s not boring though. Look at gpt-image-1, nano-banana, sora. All gone extremely viral.
People love it and so there will be more. Getting users and farming engagement is a “real usecase” I’m afraid.
People love it and so there will be more. Getting users and farming engagement is a “real usecase” I’m afraid.
October 13, 2025 at 8:38 AM
It’s not boring though. Look at gpt-image-1, nano-banana, sora. All gone extremely viral.
People love it and so there will be more. Getting users and farming engagement is a “real usecase” I’m afraid.
People love it and so there will be more. Getting users and farming engagement is a “real usecase” I’m afraid.
SCP 10110: The infinite engagement engine.
October 13, 2025 at 8:21 AM
SCP 10110: The infinite engagement engine.
Representing this as a matrix is very impractical but surprisingly it can be done arxiv.org/pdf/2410.02724
arxiv.org
October 4, 2025 at 9:33 AM
Representing this as a matrix is very impractical but surprisingly it can be done arxiv.org/pdf/2410.02724
No the technical correct version is where the “state” is just all the tokens then it’s markov in the sense that an LLM represesnts a “state transition” between n and n+1 tokens. In this framing the llm has no memory (weights stay same), it’s just moving around the space of strings of tokens.
October 4, 2025 at 9:15 AM
No the technical correct version is where the “state” is just all the tokens then it’s markov in the sense that an LLM represesnts a “state transition” between n and n+1 tokens. In this framing the llm has no memory (weights stay same), it’s just moving around the space of strings of tokens.
Science is an attempt to find and understand a pre-existing ground truth. That underlying truth does exist and isn’t a social construction.
October 4, 2025 at 9:06 AM
Science is an attempt to find and understand a pre-existing ground truth. That underlying truth does exist and isn’t a social construction.
The progress of Alibaba/Qwen is remarkable. I remember running Qwen 1.5 locally at about 1tps not too long ago.
Now they are basically frontier and OSS leader too. Clearly have the right setup internally to be able to very quickly adopt new developments. Unusual for a big corp.
Now they are basically frontier and OSS leader too. Clearly have the right setup internally to be able to very quickly adopt new developments. Unusual for a big corp.
October 2, 2025 at 9:34 AM
The progress of Alibaba/Qwen is remarkable. I remember running Qwen 1.5 locally at about 1tps not too long ago.
Now they are basically frontier and OSS leader too. Clearly have the right setup internally to be able to very quickly adopt new developments. Unusual for a big corp.
Now they are basically frontier and OSS leader too. Clearly have the right setup internally to be able to very quickly adopt new developments. Unusual for a big corp.
OpenAI “good”
October 1, 2025 at 11:59 PM
OpenAI “good”
OpenAI better
October 1, 2025 at 11:56 PM
OpenAI better
Huang is on the right side of history there. Hopefully this encourages other semi manufacturers and maybe even cloud providers to support more open stuff.
After all, lots of models and lots of users means lots of customers and lots of money…
After all, lots of models and lots of users means lots of customers and lots of money…
September 29, 2025 at 1:22 PM
Huang is on the right side of history there. Hopefully this encourages other semi manufacturers and maybe even cloud providers to support more open stuff.
After all, lots of models and lots of users means lots of customers and lots of money…
After all, lots of models and lots of users means lots of customers and lots of money…
If you do manage to get one can you post again to explain how you did it?
September 24, 2025 at 12:05 PM
If you do manage to get one can you post again to explain how you did it?
Sweet, I need to read it properly then.
August 3, 2025 at 10:57 PM
Sweet, I need to read it properly then.
Arc AGI is verifying it now so I heard. Any idea how adaptable this is to language models?
August 3, 2025 at 10:53 PM
Arc AGI is verifying it now so I heard. Any idea how adaptable this is to language models?
Agree completely. “It’s just RL”, again. The open community must not psyop ourselves for months this time…
I think the thing here will simply be a verifier for proof correctness.
Also probably a truly astronomical amount of test time tokens.
I think the thing here will simply be a verifier for proof correctness.
Also probably a truly astronomical amount of test time tokens.
July 19, 2025 at 3:47 PM
Agree completely. “It’s just RL”, again. The open community must not psyop ourselves for months this time…
I think the thing here will simply be a verifier for proof correctness.
Also probably a truly astronomical amount of test time tokens.
I think the thing here will simply be a verifier for proof correctness.
Also probably a truly astronomical amount of test time tokens.