Joint work with Sebastian Weichwald, Sebastien Lachapelle, and Luigi Gresele 🙏

For more info, check the full paper 👇

arxiv.org/abs/2410.235...

🧵Summary

A mathematical proof that, under suitable conditions, linear properties hold for either all or none of the equivalent models with same next-token distribution 😎

Exciting open questions on empirical findings remain🤔 - check Section 6 (Discussion) in the paper!

8/9

3⃣ We demonstrate what linear properties are shared by all or none LLMs.

🔥 Under mild assumptions, relational linear properties are shared!

⚠️ Parallel vectors may not be shared (they are under diversity)!

7/9

We also describe other linear properties: linear subspaces, probing, steering, based on relational strings (Paccanaro and Hinton, 2001).

💡They arise when the LLM can predict next-tokens for textual queries like: "What is the written language?" for many context strings!

6/9

2⃣ We reformulate linear properties of LLMs based on textual strings, depending on how LLMs predict next tokens

💡Parallel vectors arise from same log-ratios of next-token probs

E.g. same ratio for "easy"/"easiest" and "strong"/"strongest" in all contexts => parallel vecs

5/9

💡The extended linear equivalence underlies that two models' representations are linearly related, but in a subspace

‼️Outside that subspace, representations can differ a lot!

4/9

1⃣We extend the results by Khemakem et al. (2020), Roeder et al. (2021), removing a diversity assumption.

For the first time, we relate models with different repr. dimensions & find that repr.s of LLMs with same distribution are related by an “extended linear equivalence”!

3/9

Contributions:

1⃣ An identifiability result for LLMs
2⃣A 𝙧𝙚𝙡𝙖𝙩𝙞𝙤𝙣𝙖𝙡 reformulation of linear properties
3⃣ A proof of what properties are 𝙘𝙤𝙫𝙖𝙧𝙞𝙖𝙣𝙩 (~to Physics, cf. Villar et al. (2023)): hold for all or none of the LLMs with same next-token distribution

2/9

@yanai.bsky.social this is very interesting!! FYI, we studied the ubiquity, rather than emergence, of linear relational properties here:

openreview.net/forum?id=XCm...

Ask you to please add me :)

Then I agree 😄

What is the precise definition of feature?

I would like to ask for some back stabs to reviewer 2 🤬

I know @looselycorrect.bsky.social well enough eheh