Ramon Astudillo
@ramon-astudillo.bsky.social
Principal Research Scientist at IBM Research AI in New York. Speech, Formal/Natural Language Processing. Currently LLM post-training, structured SDG and RL. Opinions my own and non stationary.
ramon.astudillo.com
ramon.astudillo.com
Well ... it could still happen.
November 10, 2025 at 2:08 PM
Well ... it could still happen.
Pytorch seems to have won. Since architecture is not moving that much, CG toolkits lost also a little bit of importance (still very important)
November 10, 2025 at 1:06 PM
Pytorch seems to have won. Since architecture is not moving that much, CG toolkits lost also a little bit of importance (still very important)
theano was not that terrible, maybe too low level and compiled the CG slow. "scan" was *a pain*, but nowadays it's all "static" transformer bricks, so it would not be used. That also kinda killed the value of dynet.
November 10, 2025 at 4:22 AM
theano was not that terrible, maybe too low level and compiled the CG slow. "scan" was *a pain*, but nowadays it's all "static" transformer bricks, so it would not be used. That also kinda killed the value of dynet.
Isn't it more like:
analog -> digital -> normal code -> neural circuit -> language-as-code?
The neural circuit alone is pretty bad at logic, but can use the normal code for that. In exchange it gains a lot of flexibility and generalization.
analog -> digital -> normal code -> neural circuit -> language-as-code?
The neural circuit alone is pretty bad at logic, but can use the normal code for that. In exchange it gains a lot of flexibility and generalization.
November 7, 2025 at 5:22 PM
Isn't it more like:
analog -> digital -> normal code -> neural circuit -> language-as-code?
The neural circuit alone is pretty bad at logic, but can use the normal code for that. In exchange it gains a lot of flexibility and generalization.
analog -> digital -> normal code -> neural circuit -> language-as-code?
The neural circuit alone is pretty bad at logic, but can use the normal code for that. In exchange it gains a lot of flexibility and generalization.
They are actually closest to a general purpose computer! See e.g. @karpathy.bsky.social 's Software 3.0 view. IBM defines this as "Generative Computer", I prefer "Neural Computer". The idea is basically what I thought would be the GPT4 paper's title (my title was less boring)
x.com/RamonAstudil...
x.com/RamonAstudil...
November 7, 2025 at 5:08 PM
They are actually closest to a general purpose computer! See e.g. @karpathy.bsky.social 's Software 3.0 view. IBM defines this as "Generative Computer", I prefer "Neural Computer". The idea is basically what I thought would be the GPT4 paper's title (my title was less boring)
x.com/RamonAstudil...
x.com/RamonAstudil...
Oh, I made it up, that's why. I thought it was self explanatory. I was clearly wrong ...
November 7, 2025 at 4:25 PM
Oh, I made it up, that's why. I thought it was self explanatory. I was clearly wrong ...
Yeah, it has some LLM in the workflow together with conventional code.
November 7, 2025 at 3:38 PM
Yeah, it has some LLM in the workflow together with conventional code.
👆Yeah, if they had the time but there are a thousand like me, and they are probably focused on winning the big money makers like software development/maintenance automation. Will they catch up in the long term? Maybe but in the long term we are all dead and I can afford a very expensive obituary
November 7, 2025 at 3:30 PM
👆Yeah, if they had the time but there are a thousand like me, and they are probably focused on winning the big money makers like software development/maintenance automation. Will they catch up in the long term? Maybe but in the long term we are all dead and I can afford a very expensive obituary
👆Imagine, I do some SasS stuff that yields amazing gains on some vertical. Under the hood most of the value is the LLM and infra, but LLM/infra guys can't just jack up my API price. Could they eat me up by vertical integration? 👇
November 7, 2025 at 3:28 PM
👆Imagine, I do some SasS stuff that yields amazing gains on some vertical. Under the hood most of the value is the LLM and infra, but LLM/infra guys can't just jack up my API price. Could they eat me up by vertical integration? 👇
👆LLM/Infra looks pretty hard to price-segment? and it seems this is going to follow the same pattern as the internet and move so fast, we will end up with things like JavaScript. In this setting ... "Neuro Software" seems like it will be far easier to price-segment? 👇
November 7, 2025 at 3:23 PM
👆LLM/Infra looks pretty hard to price-segment? and it seems this is going to follow the same pattern as the internet and move so fast, we will end up with things like JavaScript. In this setting ... "Neuro Software" seems like it will be far easier to price-segment? 👇
enantiodromia is what happens. You can't tell the machine what "not to do" without teaching it what that thing is, which leaves it one flip switch away from happening.
November 6, 2025 at 10:20 PM
enantiodromia is what happens. You can't tell the machine what "not to do" without teaching it what that thing is, which leaves it one flip switch away from happening.