Valérie Castin
@vcastin.bsky.social
PhD student in Machine learning at Ecole Normale Supérieure, Paris
My webpage: https://vcastin.github.io/
My webpage: https://vcastin.github.io/
Reposted by Valérie Castin
A cute result from Valérie’s work is that Gaussian distributions remain closed under evolution by attentions layers, allowing one to study an ODE in the (mean, covariance) space. In particular, this enables the analysis of the “clustering of tokens” toward low-rank covariances.
February 1, 2025 at 9:54 AM
A cute result from Valérie’s work is that Gaussian distributions remain closed under evolution by attentions layers, allowing one to study an ODE in the (mean, covariance) space. In particular, this enables the analysis of the “clustering of tokens” toward low-rank covariances.