@aaditya6284.bsky.social
Pinned
Transformers employ different strategies through training to minimize loss, but how do these tradeoff and why?
Excited to share our newest work, where we show remarkably rich competitive and cooperative interactions (termed "coopetition") as a transformer learns.
Read on 🔎⏬
Excited to share our newest work, where we show remarkably rich competitive and cooperative interactions (termed "coopetition") as a transformer learns.
Read on 🔎⏬
Reposted
New work led by
@aaditya6284.bsky.social
"Strategy coopetition explains the emergence and transience of in-context learning in transformers."
We find some surprising things!! E.g. that circuits can simultaneously compete AND cooperate ("coopetition") 😯 🧵👇
@aaditya6284.bsky.social
"Strategy coopetition explains the emergence and transience of in-context learning in transformers."
We find some surprising things!! E.g. that circuits can simultaneously compete AND cooperate ("coopetition") 😯 🧵👇
March 11, 2025 at 6:18 PM
New work led by
@aaditya6284.bsky.social
"Strategy coopetition explains the emergence and transience of in-context learning in transformers."
We find some surprising things!! E.g. that circuits can simultaneously compete AND cooperate ("coopetition") 😯 🧵👇
@aaditya6284.bsky.social
"Strategy coopetition explains the emergence and transience of in-context learning in transformers."
We find some surprising things!! E.g. that circuits can simultaneously compete AND cooperate ("coopetition") 😯 🧵👇
Transformers employ different strategies through training to minimize loss, but how do these tradeoff and why?
Excited to share our newest work, where we show remarkably rich competitive and cooperative interactions (termed "coopetition") as a transformer learns.
Read on 🔎⏬
Excited to share our newest work, where we show remarkably rich competitive and cooperative interactions (termed "coopetition") as a transformer learns.
Read on 🔎⏬
March 11, 2025 at 7:13 AM
Transformers employ different strategies through training to minimize loss, but how do these tradeoff and why?
Excited to share our newest work, where we show remarkably rich competitive and cooperative interactions (termed "coopetition") as a transformer learns.
Read on 🔎⏬
Excited to share our newest work, where we show remarkably rich competitive and cooperative interactions (termed "coopetition") as a transformer learns.
Read on 🔎⏬
Reposted
What counts as in-context learning (ICL)? Typically, you might think of it as learning a task from a few examples. However, we’ve just written a perspective (arxiv.org/abs/2412.03782) suggesting interpreting a much broader spectrum of behaviors as ICL! Quick summary thread: 1/7
The broader spectrum of in-context learning
The ability of language models to learn a task from a few examples in context has generated substantial interest. Here, we provide a perspective that situates this type of supervised few-shot learning...
arxiv.org
December 10, 2024 at 6:17 PM
What counts as in-context learning (ICL)? Typically, you might think of it as learning a task from a few examples. However, we’ve just written a perspective (arxiv.org/abs/2412.03782) suggesting interpreting a much broader spectrum of behaviors as ICL! Quick summary thread: 1/7