Emanuele Marconato
@ema-ridopoco.bsky.social
Post-doc @ University of Trento. I did my PhD @ University of Trento and the University of Pisa. I like #concepts, #symbols, and #representations, but I still don't know what they are.
📍 Trento, Italy
🧵 #identifiability, #shortcuts, #interpretability
📍 Trento, Italy
🧵 #identifiability, #shortcuts, #interpretability
Pinned
🧵Why are linear properties so ubiquitous in LLM representations?
We explore this question through the lens of 𝗶𝗱𝗲𝗻𝘁𝗶𝗳𝗶𝗮𝗯𝗶𝗹𝗶𝘁𝘆:
“All or None: Identifiable Linear Properties of Next-token Predictors in Language Modeling”
Published at #AISTATS2025🌴
1/9
We explore this question through the lens of 𝗶𝗱𝗲𝗻𝘁𝗶𝗳𝗶𝗮𝗯𝗶𝗹𝗶𝘁𝘆:
“All or None: Identifiable Linear Properties of Next-token Predictors in Language Modeling”
Published at #AISTATS2025🌴
1/9
See you in San Diego 😉🏜️🏄♂️
I am happy to announce that our article "When Does Closeness in Distribution Imply Representational Similarity? An Identifiability Perspective" has been accepted at NeurIPS 2025! 🎉 arxiv.org/abs/2506.037...
Details below 👇
Details below 👇
When Does Closeness in Distribution Imply Representational Similarity? An Identifiability Perspective
When and why representations learned by different deep neural networks are similar is an active research topic. We choose to address these questions from the perspective of identifiability theory, whi...
arxiv.org
October 21, 2025 at 7:58 AM
See you in San Diego 😉🏜️🏄♂️
🧵Why are linear properties so ubiquitous in LLM representations?
We explore this question through the lens of 𝗶𝗱𝗲𝗻𝘁𝗶𝗳𝗶𝗮𝗯𝗶𝗹𝗶𝘁𝘆:
“All or None: Identifiable Linear Properties of Next-token Predictors in Language Modeling”
Published at #AISTATS2025🌴
1/9
We explore this question through the lens of 𝗶𝗱𝗲𝗻𝘁𝗶𝗳𝗶𝗮𝗯𝗶𝗹𝗶𝘁𝘆:
“All or None: Identifiable Linear Properties of Next-token Predictors in Language Modeling”
Published at #AISTATS2025🌴
1/9
June 17, 2025 at 3:12 PM
🧵Why are linear properties so ubiquitous in LLM representations?
We explore this question through the lens of 𝗶𝗱𝗲𝗻𝘁𝗶𝗳𝗶𝗮𝗯𝗶𝗹𝗶𝘁𝘆:
“All or None: Identifiable Linear Properties of Next-token Predictors in Language Modeling”
Published at #AISTATS2025🌴
1/9
We explore this question through the lens of 𝗶𝗱𝗲𝗻𝘁𝗶𝗳𝗶𝗮𝗯𝗶𝗹𝗶𝘁𝘆:
“All or None: Identifiable Linear Properties of Next-token Predictors in Language Modeling”
Published at #AISTATS2025🌴
1/9
Now in Thailand to present our paper at #AISTATS2025 🇹🇭🌴
📍Today at 3:00-6:00 pm, poster number 118!
More details here:
openreview.net/forum?id=XCm...
📍Today at 3:00-6:00 pm, poster number 118!
More details here:
openreview.net/forum?id=XCm...
All or None: Identifiable Linear Properties of Next-Token...
We analyze identifiability as a possible explanation for the ubiquity of linear properties across language models, such as the vector difference between the representations of “easy” and “easiest”...
openreview.net
May 3, 2025 at 2:36 AM
Now in Thailand to present our paper at #AISTATS2025 🇹🇭🌴
📍Today at 3:00-6:00 pm, poster number 118!
More details here:
openreview.net/forum?id=XCm...
📍Today at 3:00-6:00 pm, poster number 118!
More details here:
openreview.net/forum?id=XCm...
Only yesterday I discovered that my PhD thesis has been made public to everyone 😅
I worked three years on "Learning concepts" and I tried to spot the connection between #concepts, #symbols, and #representations, and how they're used in ML today 👾🪄
etd.adm.unipi.it/t/etd-012620...
I worked three years on "Learning concepts" and I tried to spot the connection between #concepts, #symbols, and #representations, and how they're used in ML today 👾🪄
etd.adm.unipi.it/t/etd-012620...
Tesi etd-01262025-170550
etd.adm.unipi.it
April 17, 2025 at 7:22 AM
Only yesterday I discovered that my PhD thesis has been made public to everyone 😅
I worked three years on "Learning concepts" and I tried to spot the connection between #concepts, #symbols, and #representations, and how they're used in ML today 👾🪄
etd.adm.unipi.it/t/etd-012620...
I worked three years on "Learning concepts" and I tried to spot the connection between #concepts, #symbols, and #representations, and how they're used in ML today 👾🪄
etd.adm.unipi.it/t/etd-012620...
Hey hey! We have an accepted paper at #AISTATS2025!!
Time to prepare for Thailand 🪷🏖️🌴🐒
Huge thanks to my coauthors
Luigi Gresele, Sebastian Weichwald, and @seblachap.bsky.social for all the joint effort!
More details soon 👇
arxiv.org/abs/2410.235...
Time to prepare for Thailand 🪷🏖️🌴🐒
Huge thanks to my coauthors
Luigi Gresele, Sebastian Weichwald, and @seblachap.bsky.social for all the joint effort!
More details soon 👇
arxiv.org/abs/2410.235...
January 23, 2025 at 12:18 PM
Hey hey! We have an accepted paper at #AISTATS2025!!
Time to prepare for Thailand 🪷🏖️🌴🐒
Huge thanks to my coauthors
Luigi Gresele, Sebastian Weichwald, and @seblachap.bsky.social for all the joint effort!
More details soon 👇
arxiv.org/abs/2410.235...
Time to prepare for Thailand 🪷🏖️🌴🐒
Huge thanks to my coauthors
Luigi Gresele, Sebastian Weichwald, and @seblachap.bsky.social for all the joint effort!
More details soon 👇
arxiv.org/abs/2410.235...
Don't miss the chance to learn more about our new #benchmark suite @ #NeurIPS2024
📊 New benchmarks to test concept quality learned by all kinds of models: Neural, NeSy, Concept-based, and Foundation models.
🤔 All models learn to solve the task but, beware, do they learn concepts??
Spoiler: 😱
📊 New benchmarks to test concept quality learned by all kinds of models: Neural, NeSy, Concept-based, and Foundation models.
🤔 All models learn to solve the task but, beware, do they learn concepts??
Spoiler: 😱
📣 Does your model learn high-quality #concepts, or does it learn a #shortcut?
Test it with our #NeurIPS2024 dataset & benchmark track paper!
rsbench: A Neuro-Symbolic Benchmark Suite for Concept Quality and Reasoning Shortcuts
What's the deal with rsbench? 🧵
Test it with our #NeurIPS2024 dataset & benchmark track paper!
rsbench: A Neuro-Symbolic Benchmark Suite for Concept Quality and Reasoning Shortcuts
What's the deal with rsbench? 🧵
December 11, 2024 at 12:39 PM
Don't miss the chance to learn more about our new #benchmark suite @ #NeurIPS2024
📊 New benchmarks to test concept quality learned by all kinds of models: Neural, NeSy, Concept-based, and Foundation models.
🤔 All models learn to solve the task but, beware, do they learn concepts??
Spoiler: 😱
📊 New benchmarks to test concept quality learned by all kinds of models: Neural, NeSy, Concept-based, and Foundation models.
🤔 All models learn to solve the task but, beware, do they learn concepts??
Spoiler: 😱
Reposted by Emanuele Marconato
📣 Does your model learn high-quality #concepts, or does it learn a #shortcut?
Test it with our #NeurIPS2024 dataset & benchmark track paper!
rsbench: A Neuro-Symbolic Benchmark Suite for Concept Quality and Reasoning Shortcuts
What's the deal with rsbench? 🧵
Test it with our #NeurIPS2024 dataset & benchmark track paper!
rsbench: A Neuro-Symbolic Benchmark Suite for Concept Quality and Reasoning Shortcuts
What's the deal with rsbench? 🧵
December 10, 2024 at 7:10 PM
📣 Does your model learn high-quality #concepts, or does it learn a #shortcut?
Test it with our #NeurIPS2024 dataset & benchmark track paper!
rsbench: A Neuro-Symbolic Benchmark Suite for Concept Quality and Reasoning Shortcuts
What's the deal with rsbench? 🧵
Test it with our #NeurIPS2024 dataset & benchmark track paper!
rsbench: A Neuro-Symbolic Benchmark Suite for Concept Quality and Reasoning Shortcuts
What's the deal with rsbench? 🧵
Reposted by Emanuele Marconato
The #NeurIPS experience is about to start! ✈
Drop me a line if you want to chat about #neurosymbolic reasoning #shortcuts, human-interpretable machine #concepts, logically-consistent #LLMs, or human-in-the-➰ #XAI!
See you in Vancouver!
Drop me a line if you want to chat about #neurosymbolic reasoning #shortcuts, human-interpretable machine #concepts, logically-consistent #LLMs, or human-in-the-➰ #XAI!
See you in Vancouver!
December 9, 2024 at 1:26 AM
The #NeurIPS experience is about to start! ✈
Drop me a line if you want to chat about #neurosymbolic reasoning #shortcuts, human-interpretable machine #concepts, logically-consistent #LLMs, or human-in-the-➰ #XAI!
See you in Vancouver!
Drop me a line if you want to chat about #neurosymbolic reasoning #shortcuts, human-interpretable machine #concepts, logically-consistent #LLMs, or human-in-the-➰ #XAI!
See you in Vancouver!
For those attending #NeurIPS2024 go to UNIREPS @unireps.bsky.social workshop to know more about representations similarity. Nice work led by @beatrixmgn.bsky.social 🌟
Why do NNs often learn similar representations? Existing identifiability results offer theoretical insights, but applying them in practice poses challenges.
We’ll present our new work exploring these challenges next week at @unireps #NeurIPS2024 🇨🇦🎉
openreview.net/pdf?id=SQKUZSieVg
1/
We’ll present our new work exploring these challenges next week at @unireps #NeurIPS2024 🇨🇦🎉
openreview.net/pdf?id=SQKUZSieVg
1/
openreview.net
December 6, 2024 at 4:21 PM
For those attending #NeurIPS2024 go to UNIREPS @unireps.bsky.social workshop to know more about representations similarity. Nice work led by @beatrixmgn.bsky.social 🌟
Reposted by Emanuele Marconato
🚨 Interpretable AI often means sacrificing accuracy—but what if we could have both? Most interpretable AI models, like Concept Bottleneck Models, force us to trade accuracy for interpretability.
But not anymore, due to Concept-Based Memory Reasoner (CMR)! #NeurIPS2024 (1/7)
But not anymore, due to Concept-Based Memory Reasoner (CMR)! #NeurIPS2024 (1/7)
December 4, 2024 at 8:46 AM
🚨 Interpretable AI often means sacrificing accuracy—but what if we could have both? Most interpretable AI models, like Concept Bottleneck Models, force us to trade accuracy for interpretability.
But not anymore, due to Concept-Based Memory Reasoner (CMR)! #NeurIPS2024 (1/7)
But not anymore, due to Concept-Based Memory Reasoner (CMR)! #NeurIPS2024 (1/7)
Reposted by Emanuele Marconato
We are the premier conference on #uncertainty in #AI and #ML since 1985 🧓
Hello, 🦋!
Follow us to reduce uncertainty!
Hello, 🦋!
Follow us to reduce uncertainty!
December 3, 2024 at 4:08 PM
We are the premier conference on #uncertainty in #AI and #ML since 1985 🧓
Hello, 🦋!
Follow us to reduce uncertainty!
Hello, 🦋!
Follow us to reduce uncertainty!