Marius Mosbach
LightNews LightNews
mariusmosbach.bsky.social
Marius Mosbach
@mariusmosbach.bsky.social
930 followers 280 following 28 posts
#NLP Postdoc at Mila - Quebec AI Institute & McGill University

mariusmosbach.com
Posts Replies Media Videos
mariusmosbach.bsky.social
Check out our new paper on unlearning for LLMs 🤖. We show that *not all data are unlearned equally* and argue that future work on LLM unlearning should take properties of the data to be unlearned into account. This work was lead by my intern @a-krishnan.bsky.social
🔗: arxiv.org/abs/2504.05058
Diagram illustrating a hypothesis about knowledge unlearning in language models. The left side shows a training corpus with varying frequencies of facts, such as 'Montreal is a city in Quebec' (high frequency) and 'Atlantis is a city in the ocean' (lower frequency). The center shows a language model being trained on this data, then undergoing unlearning. The right side demonstrates the 'Forget Quality' results, where the model more effectively unlearns the less frequent fact ('Atlantis is in Greece') while retaining the more frequent knowledge. Labels A, B, and C mark key points in the hypothesis: A (frequency variations in training data), B (influence of frequency), and C (unlearning effectiveness).
April 9, 2025 at 1:31 PM
mariusmosbach.bsky.social
Sparsity is all you need?
November 26, 2024 at 12:08 AM