BTW you might be interested in our TRAP paper (ACL Findings 2024), where we propose an intrinsic fingerprint method based on prompt optimization to find unique input-output pairs: bsky.app/profile/mgub...
4/4

I think that the main difference is that model fingerprinting lets the verifier pick the inputs, while an output fingerprint would make any generated output identifiable. Always happy to exchange thoughts if you're interested :)
3/

To me, your method looks like a new category: output fingerprint (something I've been thinking about for some time). Kind of like watermarking, where you have output (eg. red/green) and model (eg. instructional) watermarks.
2/

Very nice paper, congrats! I really like it.

I have a few questions:
- Is the ellipse signature robust to noise added to the logits?
- can we compute the signature if we only have access to the top-k logits?

1/

They found the universal intro for all papers:
<insert name> should be correct. But in reality, that is rarely true.

Thanks a lot Guillaume :)

I agree that there is a gap in the nb of params that a high-end device and a cheap one can run. I guess that "common consumer device" means a mid-range one. But I totally agree that they should specify the type of device: a mobile phone is quite different from a desktop computer.

My pleasure! Yes, I guess so. I agree that a moving definition can be quite annoying for research. At the same time, I think it is not specific to LMs: a large file, an heavy software, etc. 15 years ago that required a lot of resources back then, is probably be quite small for today's hardware.

There are more details in Appendix A.

This NVIDIA position paper has a clear definition of an SLM: arxiv.org/abs/2506.02153
They consider <10B.
Personally, I would not consider 13B models to be SLMs (not even 7B). They require quite a lot of resources without using aggressive efficient inference techniques (like 4 bits quantization).

This has been explored quite a lot for the task of jailbreaking an LLM (ie, adversarial examples against LLM alignment). For examples:
- arxiv.org/abs/2310.08419
- arxiv.org/abs/2312.02119
- arxiv.org/abs/2502.01633

The authors show that LLMs often give opposite answers when forced to answer vs. when not (e.g., open-ended generation). And similarly, the conclusions are highly unstable with prompting.

I agree with the need for transparency. Especially b/c the results seem highly dependent on the evaluation details. There is a really nice ACL 2024 paper about this: aclanthology.org/2024.acl-lon...

Congrats Michael! 👏🎉
Will you stay in Paris?