Many thanks to my amazing co-authors:
@tianjunzhong.bsky.social, @rjantonello.bsky.social, Gavin Mischler, Prof. Micah Goldblum and my advisor Prof. Nima Mesgarani!

#NeuroAI #LLM #NeurIPS2025 #NeurIPS

5️⃣ Takeaway:
- Raw LLM embeddings = biased toward shallow linguistic features.
- Residual disentanglement exposes the deeper, reasoning-specific representations shared by brains and models.

4️⃣Spatial pattern: reasoning even recruits visual cortex beyond classical language areas.

3️⃣ Temporal dynamics: reasoning peaks later (~350–400 ms) than shallow features.

2️⃣ We introduce the first "reasoning embedding", a disentangled representation that isolates reasoning from lexicon, syntax, and meaning.
- The disentangled representations are orthogonal to each other.

1️⃣ Why "Far from the Shallow"?
- Traditional LLM embeddings are entangled, they mix shallow linguistic features (lexicon/syntax) with deeper signals.
- This makes brain encoding studies misleading: success often comes from shallow correlations, not true semantics/reasoning alignment.

3️⃣ Unique spatial-temporal pattern of reasoning:
- Temporal dynamics: reasoning peaks later (~350–400 ms).
- Spatially: it even recruits visual cortex beyond classical language areas (IFG/STG), suggesting reasoning involves multimodal integration.
(4/6)

2️⃣ Our contribution:
- We introduce the first “reasoning embedding”, a disentangled representation that isolates reasoning from lexicon, syntax, and meaning.
- It captures variance in brain activity that shallow features can't explain, revealing a distinct neural signature for reasoning.
(3/6)