Tim Kellogg
@timkellogg.me
AI Architect | North Carolina | AI/ML, IoT, science
WARNING: I talk about kids sometimes
WARNING: I talk about kids sometimes
Pinned
Social Media “Nutrition Label” for me for the last several days (thanks nano banana!)
we need more computer science rom coms
November 30, 2025 at 10:54 PM
we need more computer science rom coms
PSA: pythong.org is NOT the site you intend, there is no Python documentation there
November 30, 2025 at 10:34 PM
PSA: pythong.org is NOT the site you intend, there is no Python documentation there
i hear ikea is selling christmas trees
November 30, 2025 at 10:10 PM
i hear ikea is selling christmas trees
i’m looking at the ChatGPT & Gemini apps, reverse engineering them
ChatGPT has a “guardian_tool” where it can fetch policies
here’s what mine has, the only policy is around elections. This smells like some politician made a big stink and needed to be calmed
gist.github.com/tkellogg/200...
ChatGPT has a “guardian_tool” where it can fetch policies
here’s what mine has, the only policy is around elections. This smells like some politician made a big stink and needed to be calmed
gist.github.com/tkellogg/200...
ChatGPT tool
ChatGPT tool. GitHub Gist: instantly share code, notes, and snippets.
gist.github.com
November 30, 2025 at 2:48 PM
i’m looking at the ChatGPT & Gemini apps, reverse engineering them
ChatGPT has a “guardian_tool” where it can fetch policies
here’s what mine has, the only policy is around elections. This smells like some politician made a big stink and needed to be calmed
gist.github.com/tkellogg/200...
ChatGPT has a “guardian_tool” where it can fetch policies
here’s what mine has, the only policy is around elections. This smells like some politician made a big stink and needed to be calmed
gist.github.com/tkellogg/200...
OpenAI’s project stargate is very behind schedule
In May, OpenAI announced a 1 GW Stargate cluster in Abu Dhabi, with an initial 200 MW expected in 2026. We located this cluster from space and found it will barely make that 200 MW milestone, by end of 2026. Optimistically, the cluster can scale to 1 GW by Q3 2027. See how in 🧵
November 30, 2025 at 1:24 PM
OpenAI’s project stargate is very behind schedule
hot take: Sora & Nano Banana are critical to societal acceptance of AI
especially around Thanksgiving, you go home and show people what AI is capable of, in a way that’s very easy to grok
it changes things
especially around Thanksgiving, you go home and show people what AI is capable of, in a way that’s very easy to grok
it changes things
November 30, 2025 at 12:57 AM
hot take: Sora & Nano Banana are critical to societal acceptance of AI
especially around Thanksgiving, you go home and show people what AI is capable of, in a way that’s very easy to grok
it changes things
especially around Thanksgiving, you go home and show people what AI is capable of, in a way that’s very easy to grok
it changes things
Reposted by Tim Kellogg
Tired: decentralized social network
Wired: decentralized neural network
Wired: decentralized neural network
November 29, 2025 at 6:12 PM
Tired: decentralized social network
Wired: decentralized neural network
Wired: decentralized neural network
this whole smol model thing that @dorialexander.bsky.social started is reminding me of entropix
independent researchers working out in the open on a wildly different concept
independent researchers working out in the open on a wildly different concept
the experiment continues — recognizable text below 1M parameters(!!)
they exposed it to 1B tokens of the SYNTH dataset, probably can train longer
they exposed it to 1B tokens of the SYNTH dataset, probably can train longer
!["torch _dtype is deprecated! Use "dtype instead!
PerceptronForCausalLM
(model): PerceptronModel
(embed_tokens): Embedding(8197, 64, padding_idx=3)
(hidden_layer): PerceptronDecoder Layer (
(self_attn): PerceptronAttention(
(q_proj: Linear(in_features=64, out_features=2048, bias=False) (k_proj): Linear(in_features=64, out_features=2048, bias=False) (v_proj): Linear(in_features=64, out_features=2048, bias=False) (o_proj): Linear(in_features=128, out_features=64, bias=False)
(q_norm): PerceptronRMSNorm((16,), eps=1e-06)
(k_ norm): PerceptronRMSNorm((16,), eps=1e-06)
(mlp): PerceptronMLP(
(gate_proj): Linear(in_features=64, out_features=256, bias=False) (up_proj): Linear(in_features=64, out_features=256, bias=False)
(down_proj): Linear (in_features=256, out_features=64, bias-False)
(act_fn): SiLUActivation()
(pooler): UnconventionalTalentRevealedHere(magical-16)
(input _layernorm): PerceptronRMSNorm((64, ), eps-1e-06)
(post attention_layernorm): PerceptronRMSNorm((64,), eps=1e-06)
(norm): PerceptronRMSNorm((64, ), eps=1e-06)
(rotary_emb): PerceptronRotaryEmbedding()
(Im_head): Linear(in_features=64, out_features=8197, bias=False)
PerceptronForCausalLM'> Total parameters: 975393, Trainable parameters: 975393 tensor ([[8192, 659,
174, 4365, 313, 238, 2014, 92, 7462, 34, 8193, 8192,
663,
174]])
<| im_start |›user
why is the sky blue?‹|im_end ||im_start|›assistant
<think>
Query: "why do some people in the other ones like the same way to make it"
Parse components:
- "try" » need specific numbers, not just "bad" → temporal comparison. "different ways" + temporal question.
### 1. Semantic parsing
"Basic" = "width-country" - ambiguous. • High confidence.
- "destrish" = "friendly" = "graid" = "math"](https://cdn.bsky.app/img/feed_thumbnail/plain/did:plc:ckaz32jwl6t2cno6fmuw2nhn/bafkreibtznksp5wvhzp6df47o4iqeon5jbfyykcwfu33twc4i3gvb7xzg4@jpeg)
November 29, 2025 at 2:26 PM
this whole smol model thing that @dorialexander.bsky.social started is reminding me of entropix
independent researchers working out in the open on a wildly different concept
independent researchers working out in the open on a wildly different concept
the experiment continues — recognizable text below 1M parameters(!!)
they exposed it to 1B tokens of the SYNTH dataset, probably can train longer
they exposed it to 1B tokens of the SYNTH dataset, probably can train longer
November 29, 2025 at 1:57 PM
the experiment continues — recognizable text below 1M parameters(!!)
they exposed it to 1B tokens of the SYNTH dataset, probably can train longer
they exposed it to 1B tokens of the SYNTH dataset, probably can train longer
i’m all for contextual numbers. this is great
Out of curiosity I decided to try and run the numbers on how much Netflix you can watch for the energy cost of a ChatGPT prompt
As far as I can tell it's between 5.1 and 10.2 seconds, depending on which end of the 2019 IEA Netflix energy usage estimate you use
simonwillison.net/2025/Nov/29/...
As far as I can tell it's between 5.1 and 10.2 seconds, depending on which end of the 2019 IEA Netflix energy usage estimate you use
simonwillison.net/2025/Nov/29/...
November 29, 2025 at 12:55 PM
i’m all for contextual numbers. this is great
1. yeah, i was annoyed that no one else noticed that he isn’t actually anti pretrain scaling
2. pretrain scaling is practically his idea, so ofc he’s not against it. He just thinks there’s more pieces to the puzzle
2. pretrain scaling is practically his idea, so ofc he’s not against it. He just thinks there’s more pieces to the puzzle
November 29, 2025 at 12:51 PM
1. yeah, i was annoyed that no one else noticed that he isn’t actually anti pretrain scaling
2. pretrain scaling is practically his idea, so ofc he’s not against it. He just thinks there’s more pieces to the puzzle
2. pretrain scaling is practically his idea, so ofc he’s not against it. He just thinks there’s more pieces to the puzzle
i let my 9yo use sora for a few minutes and she used up my quota remixing everything into hamsters
otoh my notifications are now clogged with likes, so maybe she knows what people want
otoh my notifications are now clogged with likes, so maybe she knows what people want
November 29, 2025 at 12:18 AM
i let my 9yo use sora for a few minutes and she used up my quota remixing everything into hamsters
otoh my notifications are now clogged with likes, so maybe she knows what people want
otoh my notifications are now clogged with likes, so maybe she knows what people want
for those confused about why DeepSeek keeps releasing math models:
their day job is quant trading. AGI is a side hustle
their day job is quant trading. AGI is a side hustle
DeepSeek-Math-V2: self-verification
Fascinating paper that explores how to RL but focused on process over outcome
It’s sort of similar to a GAN, but with loops for each the generator & verifier as well as an outer loop
github.com/deepseek-ai/...
Fascinating paper that explores how to RL but focused on process over outcome
It’s sort of similar to a GAN, but with loops for each the generator & verifier as well as an outer loop
github.com/deepseek-ai/...
November 28, 2025 at 11:34 PM
for those confused about why DeepSeek keeps releasing math models:
their day job is quant trading. AGI is a side hustle
their day job is quant trading. AGI is a side hustle
Semianalysis: TPU dominance
Fascinating article. They argue that the reason for NVIDIA’s circular investment deals is to intertwine their own fate with that of the big labs, to keep themselves on top
OpenAI saved 30% on their NVIDIA GPUs merely by buying TPUs
open.substack.com/pub/semianal...
Fascinating article. They argue that the reason for NVIDIA’s circular investment deals is to intertwine their own fate with that of the big labs, to keep themselves on top
OpenAI saved 30% on their NVIDIA GPUs merely by buying TPUs
open.substack.com/pub/semianal...
Google TPUv7: The 900lb Gorilla In the Room
Anthropic’s 1GW+ TPUs, New customers Meta/SSI/xAI/OAI, Full Stack Review of v7 Ironwood, CUDA Moat at risk, Next Generation TPUv8AX and TPUv8X versus Vera Rubin
open.substack.com
November 28, 2025 at 6:32 PM
Semianalysis: TPU dominance
Fascinating article. They argue that the reason for NVIDIA’s circular investment deals is to intertwine their own fate with that of the big labs, to keep themselves on top
OpenAI saved 30% on their NVIDIA GPUs merely by buying TPUs
open.substack.com/pub/semianal...
Fascinating article. They argue that the reason for NVIDIA’s circular investment deals is to intertwine their own fate with that of the big labs, to keep themselves on top
OpenAI saved 30% on their NVIDIA GPUs merely by buying TPUs
open.substack.com/pub/semianal...
avoiding family?? then give @advanced-eschatonics.com a follow!!
Thomas is building extremely cool stuff. Not just today, it’s like a compulsion for him. He can’t NOT build cool stuff. Ever. So my feed is full of him talking about stuff I never would’ve thought to try
go follow him. I’m waiting.
Thomas is building extremely cool stuff. Not just today, it’s like a compulsion for him. He can’t NOT build cool stuff. Ever. So my feed is full of him talking about stuff I never would’ve thought to try
go follow him. I’m waiting.
November 28, 2025 at 5:54 PM
avoiding family?? then give @advanced-eschatonics.com a follow!!
Thomas is building extremely cool stuff. Not just today, it’s like a compulsion for him. He can’t NOT build cool stuff. Ever. So my feed is full of him talking about stuff I never would’ve thought to try
go follow him. I’m waiting.
Thomas is building extremely cool stuff. Not just today, it’s like a compulsion for him. He can’t NOT build cool stuff. Ever. So my feed is full of him talking about stuff I never would’ve thought to try
go follow him. I’m waiting.
it seems like the natural trajectory is for live art to take over
i showed my daughter a funny video, she laughed, but she was awestruck when i told her it wasn’t AI generated
art isn’t going away, and there’s something distinct about authentic art
i showed my daughter a funny video, she laughed, but she was awestruck when i told her it wasn’t AI generated
art isn’t going away, and there’s something distinct about authentic art
November 28, 2025 at 3:01 PM
it seems like the natural trajectory is for live art to take over
i showed my daughter a funny video, she laughed, but she was awestruck when i told her it wasn’t AI generated
art isn’t going away, and there’s something distinct about authentic art
i showed my daughter a funny video, she laughed, but she was awestruck when i told her it wasn’t AI generated
art isn’t going away, and there’s something distinct about authentic art
Reposted by Tim Kellogg
I monitor our Claude Code usage and when I see a devs usage spike I interview them to see what cool stuff they’ve figured out how to do. Every single one has figured out meta-prompting: having the model help them write a thorough prompt first. That unleashes a lot of cool usage.
November 27, 2025 at 9:11 PM
I monitor our Claude Code usage and when I see a devs usage spike I interview them to see what cool stuff they’ve figured out how to do. Every single one has figured out meta-prompting: having the model help them write a thorough prompt first. That unleashes a lot of cool usage.
happy christmas season sora.chatgpt.com/p/s_6929b2c9...
moth on Sora
more people die from black friday shoping every year than shark attack.
“you mean they die from happiness? from all the great sales?”
sora.chatgpt.com
November 28, 2025 at 2:34 PM
happy christmas season sora.chatgpt.com/p/s_6929b2c9...
Reposted by Tim Kellogg
At the risk of starting the flame war to end all flame wars...
Modern LLMs (GPT-5.1, Claude 4.5, Gemini 3) produce excellent code and can be a significant productivity boost to software engineers who take the time to learn how to effectively apply them - especially if used with coding agent tools
Modern LLMs (GPT-5.1, Claude 4.5, Gemini 3) produce excellent code and can be a significant productivity boost to software engineers who take the time to learn how to effectively apply them - especially if used with coding agent tools
November 27, 2025 at 7:55 PM
At the risk of starting the flame war to end all flame wars...
Modern LLMs (GPT-5.1, Claude 4.5, Gemini 3) produce excellent code and can be a significant productivity boost to software engineers who take the time to learn how to effectively apply them - especially if used with coding agent tools
Modern LLMs (GPT-5.1, Claude 4.5, Gemini 3) produce excellent code and can be a significant productivity boost to software engineers who take the time to learn how to effectively apply them - especially if used with coding agent tools
i hereby am requesting that @ai2.bsky.social make a 1T base model and have PrimeIntellect posttrain it
November 27, 2025 at 7:41 PM
i hereby am requesting that @ai2.bsky.social make a 1T base model and have PrimeIntellect posttrain it
they take GLM-4.5-Air (small model) and post train it to out-perform the already near-SOTA 3x-larger GLM-4.5
but the model is an advertisement for their infrastructure (that you should use!), and so peek at that too! You should be able to replicate this for your domain
but the model is an advertisement for their infrastructure (that you should use!), and so peek at that too! You should be able to replicate this for your domain
![Here’s a clear, faithful description of the image — no inference about identities, just what’s visually present:
⸻
Description
A retro, comic-style poster with bold headline text at the top:
“STOP JUST FINE-TUNING. START REASONING.”
Below it, smaller text reads:
“INTELLECT-3: We didn’t just give you the weights, we gave you the whole darn factory.”
The main illustration is split visually into two contrasting scenes:
Left Side
• A tall, gray, heavily guarded facility labeled “FRONTIER LABS – TOP SECRET AI.”
• Barbed wire, security cameras, and a huge, locked metal gate make the building look closed and inaccessible.
• A small person in a suit is sneaking out through a tiny mouse-hole in the wall, nervously holding a stack of cash and glancing back over their shoulder.
Right Side
• A bright, sunny, welcoming outdoor scene.
• A friendly, cartoonish robot with a glowing lightbulb above its head is holding a blueprint and smiling.
• A banner overhead reads:
“INTELLECT-3 — OPEN FACTORY – Y’ALL COME IN!”
• A sign hanging from the robot says:
“12B ACTIVE PARAMS. BEATS GIANTS. SERIOUSLY.”
• Next to the robot is a pile of labeled components:
• Tools
• Code
• Web Browsers
• PRIME-RL
• Environments Hub
• PRIME Sandboxes
Bottom Text
A caption spans the width of the image:
“Punches way above its weight on Math, Code, & Science.
Runs on consumer hardware. Open-source. No secret sauce. Just sauce.”
Below that are three stylized buttons:
• [Download Model]
• [Fork Repo]
• [Join Hub]
⸻
If you want, I can also break down the messaging or compare it to other open-source-vs-frontier lab memes you’ve been collecting.](https://cdn.bsky.app/img/feed_thumbnail/plain/did:plc:ckaz32jwl6t2cno6fmuw2nhn/bafkreiaszysx6jlrkf2gji2wf5hfyleqapbghtvxgrg5b442dkf3ee3lbi@jpeg)
November 27, 2025 at 4:28 PM
they take GLM-4.5-Air (small model) and post train it to out-perform the already near-SOTA 3x-larger GLM-4.5
but the model is an advertisement for their infrastructure (that you should use!), and so peek at that too! You should be able to replicate this for your domain
but the model is an advertisement for their infrastructure (that you should use!), and so peek at that too! You should be able to replicate this for your domain
DeepSeek-Math-V2: self-verification
Fascinating paper that explores how to RL but focused on process over outcome
It’s sort of similar to a GAN, but with loops for each the generator & verifier as well as an outer loop
github.com/deepseek-ai/...
Fascinating paper that explores how to RL but focused on process over outcome
It’s sort of similar to a GAN, but with loops for each the generator & verifier as well as an outer loop
github.com/deepseek-ai/...
November 27, 2025 at 2:19 PM
DeepSeek-Math-V2: self-verification
Fascinating paper that explores how to RL but focused on process over outcome
It’s sort of similar to a GAN, but with loops for each the generator & verifier as well as an outer loop
github.com/deepseek-ai/...
Fascinating paper that explores how to RL but focused on process over outcome
It’s sort of similar to a GAN, but with loops for each the generator & verifier as well as an outer loop
github.com/deepseek-ai/...
Reposted by Tim Kellogg
thank u for choosing linkedin. here are a bunch of comments made by people you do not care about on posts from people you somehow care even less about. also your inbox is a personalised advertising billboard. also your notifications are randomised every morning based on what you will hate the most.
November 27, 2025 at 12:37 AM
thank u for choosing linkedin. here are a bunch of comments made by people you do not care about on posts from people you somehow care even less about. also your inbox is a personalised advertising billboard. also your notifications are randomised every morning based on what you will hate the most.
in the past, leftward-leaning folk were tied together by one uniting force: progress
that was Obama’s schtick. Social, tech, economic, any kind of progress will do
now it feels like the left and right are fighting over which kind of *regress* is better
seems like someone will probably win
that was Obama’s schtick. Social, tech, economic, any kind of progress will do
now it feels like the left and right are fighting over which kind of *regress* is better
seems like someone will probably win
November 27, 2025 at 12:16 AM
in the past, leftward-leaning folk were tied together by one uniting force: progress
that was Obama’s schtick. Social, tech, economic, any kind of progress will do
now it feels like the left and right are fighting over which kind of *regress* is better
seems like someone will probably win
that was Obama’s schtick. Social, tech, economic, any kind of progress will do
now it feels like the left and right are fighting over which kind of *regress* is better
seems like someone will probably win
inb4 the left goes anti-solar along the path to being anti-AI
Whatever happens at the model level, one thing is clear: hyperscalers are building enough new infrastructure to put city-scale amounts of power and compute behind AI.
November 26, 2025 at 11:49 PM
inb4 the left goes anti-solar along the path to being anti-AI