Our paper: arxiv.org/pdf/2505.23368
Our code: github.com/mainlp/CoT2EL

Thank you to my wonderful co-authors,
@janetlauyeung.bsky.social, Anna Korhonen, and @barbaraplank.bsky.social. Also to @mainlp.bsky.social , @cislmu.bsky.social @munichcenterml.bsky.social

See you in Suzhou!
#NLP #EMNLP2025

Matching exact probabilities for HLV is unstable. So, we propose a more robust rank-based evaluation that checks preference order. Our combined method outperforms baselines on 3 datasets that exhibit human label variation, showing it better aligns with diverse human perspectives.

Instead of unnatural post-hoc explanations, we look forward. A model's CoT already contains rationales for all options. We introduce CoT2EL, a pipeline that uses linguistic discourse segmenters to extract these high-quality, faithful units to explore human label variation.

🌍 Broader impact:
Our approach makes capturing disagreement scalable, helping build datasets that reflect real-world ambiguity—without requiring tons of human-written explanations.
Open-sourcing:
📂 github.com/mainlp/MJD-E...

🧠 What’s this about?
Human annotations often disagree. Instead of collapsing disagreement into a single label, we model Human Judgment Distributions — how likely humans are to choose each label in NLI tasks.
Capturing this is crucial for interpretability and uncertainty in NLP.

🔗 Paper link: arxiv.org/abs/2412.13942
🙏 Huge thanks to our collaborators Logan Siyao Peng, @barbaraplank.bsky.social, Anna Korhonen from @mainlp.bsky.social, @lmumuenchen.bsky.social, ‪@cambridgeltl.bsky.social ‬