Dennis Ulmer @EMNLP
@dnnslmr.bsky.social
Postdoctoral researcher at the Institute for Logic, Language and Computation at the University of Amsterdam.
Previously PhD Student at NLPNorth at the IT University of Copenhagen, with internships at AWS, Parameter Lab, Pacmed.
dennisulmer.eu
Previously PhD Student at NLPNorth at the IT University of Copenhagen, with internships at AWS, Parameter Lab, Pacmed.
dennisulmer.eu
Just had a good laugh about how this linkedin poster portrays this paper's method vs the authors themselves
November 18, 2025 at 8:07 PM
Just had a good laugh about how this linkedin poster portrays this paper's method vs the authors themselves
AI researchers when overleaf is down and they rediscover life outside of academia
May 14, 2025 at 8:18 AM
AI researchers when overleaf is down and they rediscover life outside of academia
I ascribe the success mostly to what might my nicest figure. Took an eternity to write, was rejected twice, and every new paper that came out during the time of writing that I had to read it felt like my last nail (but I didn't learn since I am working on another survey rn)
April 22, 2025 at 8:08 AM
I ascribe the success mostly to what might my nicest figure. Took an eternity to write, was rejected twice, and every new paper that came out during the time of writing that I had to read it felt like my last nail (but I didn't learn since I am working on another survey rn)
In an earlier work in 1976, he argues for these terms based on some ideas of Poisson (see second screenshot). But I think (???) that this might be the time these terms were defined (also see how aleatoric was actually named aleatory), since I cannot find any occurrences of the phrases before 1975.
April 4, 2025 at 1:25 PM
In an earlier work in 1976, he argues for these terms based on some ideas of Poisson (see second screenshot). But I think (???) that this might be the time these terms were defined (also see how aleatoric was actually named aleatory), since I cannot find any occurrences of the phrases before 1975.
To give my own answer (so far): I found this paper by Shafer from 1978 describing how some of these ideas go back to the 18th century to Bernoulli & Lambert (but weren't even called probabilities back then, and later only "objective" and "subjective" probabilities)
April 4, 2025 at 1:25 PM
To give my own answer (so far): I found this paper by Shafer from 1978 describing how some of these ideas go back to the 18th century to Bernoulli & Lambert (but weren't even called probabilities back then, and later only "objective" and "subjective" probabilities)
Very happy to have learned about my old @nlpnorth.bsky.social colleague @mjjzha.bsky.social et al.'s work on SnakModel, a new language model for Danish!
December 19, 2024 at 10:20 AM
Very happy to have learned about my old @nlpnorth.bsky.social colleague @mjjzha.bsky.social et al.'s work on SnakModel, a new language model for Danish!
Took a bit but finally 🥹🥹
December 17, 2024 at 11:06 AM
Took a bit but finally 🥹🥹
Big congratulations to Dr. @jumelet.bsky.social for obtaining his PhD today and crafting a beautiful thesis full of original and insightful work!! 🎉 arxiv.org/pdf/2411.16433?
December 10, 2024 at 3:07 PM
Big congratulations to Dr. @jumelet.bsky.social for obtaining his PhD today and crafting a beautiful thesis full of original and insightful work!! 🎉 arxiv.org/pdf/2411.16433?
I have been made aware of this amazing paper recently from glaciology that is exactly about this: journals.sagepub.com/doi/pdf/10.1...
December 10, 2024 at 2:08 PM
I have been made aware of this amazing paper recently from glaciology that is exactly about this: journals.sagepub.com/doi/pdf/10.1...
Life update 🎉: Today is my first day starting as a postoctoral researcher at Institute for Logic, Language and Computation at the University of Amsterdam under Ivan Titov!
November 1, 2024 at 2:00 PM
Life update 🎉: Today is my first day starting as a postoctoral researcher at Institute for Logic, Language and Computation at the University of Amsterdam under Ivan Titov!
My dissertation "On Uncertainty In Natural Language Processing" is on arxiv! 🥳🎓
Check out my monograph for a background section summarizing statistical & linguistic views on UQ, a broad overview over methods used in ML & NLProc and so much more!
arxiv.org/pdf/2410.03446
Check out my monograph for a background section summarizing statistical & linguistic views on UQ, a broad overview over methods used in ML & NLProc and so much more!
arxiv.org/pdf/2410.03446
October 7, 2024 at 2:46 PM
My dissertation "On Uncertainty In Natural Language Processing" is on arxiv! 🥳🎓
Check out my monograph for a background section summarizing statistical & linguistic views on UQ, a broad overview over methods used in ML & NLProc and so much more!
arxiv.org/pdf/2410.03446
Check out my monograph for a background section summarizing statistical & linguistic views on UQ, a broad overview over methods used in ML & NLProc and so much more!
arxiv.org/pdf/2410.03446
We also ablated the inputs to the auxiliary model: Training on the question alone already produces quite good results, but they can sometimes be further improved through using chain-of-thought answers or the LLMs' own assessment of uncertainty! [5/6]
March 12, 2024 at 4:27 PM
We also ablated the inputs to the auxiliary model: Training on the question alone already produces quite good results, but they can sometimes be further improved through using chain-of-thought answers or the LLMs' own assessment of uncertainty! [5/6]
We beat a variety of baselines, for instance (Platt-scaled) sequence likelihoods, or verbalized uncertainties. [4/6]
March 12, 2024 at 4:27 PM
We beat a variety of baselines, for instance (Platt-scaled) sequence likelihoods, or verbalized uncertainties. [4/6]
In our experiments on TriviaQA and CoQA for both Vicuna v1.5 7B and GPT-3.5, we find out method to achieve the best Brier scores and misprediction AUROCs along with some very competitive calibration errors! [3/6]
March 12, 2024 at 4:26 PM
In our experiments on TriviaQA and CoQA for both Vicuna v1.5 7B and GPT-3.5, we find out method to achieve the best Brier scores and misprediction AUROCs along with some very competitive calibration errors! [3/6]
We present APRICOT 🍑, a method for LLM confidence estimation that creates confidence targets in an unsupervised manner and then trains an auxiliary model to directly predict them from the input question and the LLM answer in text form. [2/6]
March 12, 2024 at 4:26 PM
We present APRICOT 🍑, a method for LLM confidence estimation that creates confidence targets in an unsupervised manner and then trains an auxiliary model to directly predict them from the input question and the LLM answer in text form. [2/6]
Obtaining calibrated confidence scores from LLMs is hard, especially for black-box models. So, can we maybe predict them directly from the generated text? 🤔 Internship work at
Parameter Lab with Martin Gubri, Sangdoo Yun, Hwaran Lee, Seong Joon Oh! arxiv.org/abs/2403.059... [1/6]
Parameter Lab with Martin Gubri, Sangdoo Yun, Hwaran Lee, Seong Joon Oh! arxiv.org/abs/2403.059... [1/6]
March 12, 2024 at 4:24 PM
Obtaining calibrated confidence scores from LLMs is hard, especially for black-box models. So, can we maybe predict them directly from the generated text? 🤔 Internship work at
Parameter Lab with Martin Gubri, Sangdoo Yun, Hwaran Lee, Seong Joon Oh! arxiv.org/abs/2403.059... [1/6]
Parameter Lab with Martin Gubri, Sangdoo Yun, Hwaran Lee, Seong Joon Oh! arxiv.org/abs/2403.059... [1/6]
We conduct experiments in machine translation and text generation and demonstrate that this yields tighter prediction sets with better coverage! Check the paper for more analyses and discussions 🙂 [4/4]
February 2, 2024 at 3:38 PM
We conduct experiments in machine translation and text generation and demonstrate that this yields tighter prediction sets with better coverage! Check the paper for more analyses and discussions 🙂 [4/4]
Interested in how to apply Conformal Prediction to NLG? Check out our work on "Non-Exchangeable Conformal Language Generation with Nearest Neighbors" with
Chryssa Zerva and André Martins accepted to #EACL2024 Findings! [1/4] browse.arxiv.org/abs/2402.00707
Chryssa Zerva and André Martins accepted to #EACL2024 Findings! [1/4] browse.arxiv.org/abs/2402.00707
February 2, 2024 at 3:37 PM
Interested in how to apply Conformal Prediction to NLG? Check out our work on "Non-Exchangeable Conformal Language Generation with Nearest Neighbors" with
Chryssa Zerva and André Martins accepted to #EACL2024 Findings! [1/4] browse.arxiv.org/abs/2402.00707
Chryssa Zerva and André Martins accepted to #EACL2024 Findings! [1/4] browse.arxiv.org/abs/2402.00707
Since we chose existing persona descriptions from the LIGHT dataset by Urbanek et al., this made also for some extremely entertaining dialogues! [6/7]
January 11, 2024 at 3:08 PM
Since we chose existing persona descriptions from the LIGHT dataset by Urbanek et al., this made also for some extremely entertaining dialogues! [6/7]
Furthermore, we evaluate our finetuned agents through automated metrics and human evaluation and analyze which properties of the bootstrapped dialogues are indicative of good quality. [5/7]
January 11, 2024 at 3:07 PM
Furthermore, we evaluate our finetuned agents through automated metrics and human evaluation and analyze which properties of the bootstrapped dialogues are indicative of good quality. [5/7]
We also develop an automated way to assess how successful the dialogue was which strongly correlates with manual assessments. This helps to filter out dialogues of low quality that are not useful for finetuning. [4/7]
January 11, 2024 at 3:07 PM
We also develop an automated way to assess how successful the dialogue was which strongly correlates with manual assessments. This helps to filter out dialogues of low quality that are not useful for finetuning. [4/7]
We show that by assigning personas to two language models and careful prompting, we can bootstrap dialogues through a procedure we call 𝘴𝘦𝘭𝘧-𝘵𝘢𝘭𝘬. [3/7]
January 11, 2024 at 3:06 PM
We show that by assigning personas to two language models and careful prompting, we can bootstrap dialogues through a procedure we call 𝘴𝘦𝘭𝘧-𝘵𝘢𝘭𝘬. [3/7]