@miac27.bsky.social
9.2K followers 15K following 1.7K posts
Making Connections–Making a Difference. Listening - Learning - Living. …Climate Change is Real!! The greatest achievements are those which benefit others. And, never surrender…
Posts Media Videos Starter Packs
Reposted
jkz12769.bsky.social
The House Democrats last night brought the Arizona-elect representative into the Speaker office chanting "swear her in". Her election was certified but Johnson refused to swear her in because of the Epstein files. Mike Johnson needs to be Impeached for treason at this point.
Reposted
Reposted
just-jack-1.bsky.social
Kudos to dozens of airports across the country that are refusing to play state propaganda.
Reposted
muellershewrote.com
This isn’t a peace plan. It’s a ceasefire. Why is the media calling it a peace plan?
Reposted
ronfilipkowski.bsky.social
They showed his neck vagina. That’s what he’s really upset about.
Reposted
Reposted
billjonesjr.bsky.social
Sunset over Jersey City with the Hudson River in the foreground, New Jersey, 2025

#photography #AlphabetChallenge #WeekPforPink #EastCoastKin #urbanphotography
Reposted
briantylercohen.bsky.social
BREAKING: President Obama comes out in favor of Prop 50 in California.

California, help protect democracy and return your ballots today!
Reposted
barackobama.bsky.social
California, this November 4th, the whole country is counting on you.

Prop 50 puts our elections back on a level playing field, preserves independent redistricting over the long term, and lets the people decide.

So return your ballot today. Vote yes on 50.
Reposted
fo5tered.bsky.social
A Dunnock in the Park last Sunday morning 🪶 #birds
Reposted
timkellogg.me
Karpathy: nanochat

A small training+inference pipeline for creating your own LLM from scratch

$100 will get you a somewhat functional model

$1000 is more coherent & solves math

detailed walkthrough: github.com/karpathy/nan...

repo: github.com/karpathy/nan...
Andrej Karpathy & @karpathy
X.com
Excited to release new repo: nanochat! (it's among the most unhinged I've written).
Unlike my earlier similar repo nanoGPT which only covered pretraining, nanochat is a minimal, from scratch, full-stack training/inference pipeline of a simple ChatGPT clone in a single, dependency-minimal codebase. You boot up a cloud GPU box, run a single script and in as little as 4 hours later you can talk to your own LLM in a ChatGPT-like web Ul.
It weighs ~8,000 lines of imo quite clean code to:
- Train the tokenizer using a new Rust implementation
- Pretrain a Transformer LLM on FineWeb, evaluate CORE score across a number of metrics
- Midtrain on user-assistant conversations from SmolTalk, multiple choice questions, tool use.
- SFT, evaluate the chat model on world knowledge multiple choice (ARC-E/C, MMLU), math (GSM8K), code (HumanEval)
- RL the model optionally on GSM8K with
IPDDOI - RL the model optionally on GSM8K with
"GRPO"
- Efficient inference the model in an Engine with
KV cache, simple prefill/ decode, tool use (Python interpreter in a lightweight sandbox), talk to it over CLI or ChatGPT-like WebUl.
- Write a single markdown report card, summarizing and gamifying the whole thing.
Even for as low as ~$100 in cost (~4 hours on an
8XH100 node), you can train a little ChatGPT clone that you can kind of talk to, and which can write stories/poems, answer simple questions.
About ~12 hours surpasses GPT-2 CORE metric.
As you further scale up towards ~$1000 (~41.6 hours of training), it quickly becomes a lot more coherent and can solve simple math/code problems and take multiple choice tests. E.g. a depth 30 model trained for 24 hours (this is about equal to FLOPs of GPT-3 Small 125M and 1/1000th of GPT-3) gets into 40s on MMLU and
70s on ARC-Easy, 20s on GSM8K, etc.
My goal is to get the full "strong baseline" stack into one cohesive, minimal, readable, hackable, maximally forkable repo. nanochat will be the capstone project of LLM101n (which is still being developed). I think it also has potential to grow developed). I think it also has potential to grow into a research harness, or a benchmark, similar to nanoGPT before it. It is by no means finished, tuned or optimized (actually I think there's likely quite a bit of low-hanging fruit), but I think it's at a place where the overall skeleton is ok enough that it can go up on GitHub where all the parts of it can be improved.
Link to repo and a detailed walkthrough of the nanochat speedrun is in the reply.
nanochat