Daniel Fried
@daniel-fried.bsky.social
Assistant prof at LTI CMU; Research scientist at Meta AI. Working on NLP: language interfaces, applied pragmatics, language-to-code, grounding. https://dpfried.github.io/
I'm excited about Andy's work -- generating scenarios that force LLMs to choose between conflicting values, allowing us to see which values they prioritize. Might be used for training in the future! We also show the importance of open-ended (vs multiple choice) evaluation.
🚨New Paper: LLM developers aim to align models with values like helpfulness or harmlessness. But when these conflict, which values do models choose to support? We introduce ConflictScope, a fully-automated evaluation pipeline that reveals how models rank values under conflict.
(📷 xkcd)
(📷 xkcd)
October 3, 2025 at 4:13 PM
I'm excited about Andy's work -- generating scenarios that force LLMs to choose between conflicting values, allowing us to see which values they prioritize. Might be used for training in the future! We also show the importance of open-ended (vs multiple choice) evaluation.
Reposted by Daniel Fried
The IVADO #Bootcamp marked the launch of the Thematic Semester on Autonomous #LLM Agents last week at the MIL Campus of @umontreal.ca. Over 4 days, researchers, experts, and #AI enthusiasts gathered for conferences, tutorials, and rich discussions, laying the groundwork for our next two workshops.
August 19, 2025 at 2:38 PM
The IVADO #Bootcamp marked the launch of the Thematic Semester on Autonomous #LLM Agents last week at the MIL Campus of @umontreal.ca. Over 4 days, researchers, experts, and #AI enthusiasts gathered for conferences, tutorials, and rich discussions, laying the groundwork for our next two workshops.
Excited to be presenting two of our papers at #ICML2025 and workshops, today through Saturday! Topics are memory for agents, and constructing coding environments for training & evaluation. See links below:
July 16, 2025 at 6:30 PM
Excited to be presenting two of our papers at #ICML2025 and workshops, today through Saturday! Topics are memory for agents, and constructing coding environments for training & evaluation. See links below:
Reposted by Daniel Fried
Happy to announce the first workshop on Pragmatic Reasoning in Language Models — PragLM @ COLM 2025! 🎉
How do LLMs engage in pragmatic reasoning, and what core pragmatic capacities remain beyond their reach?
🌐 sites.google.com/berkeley.edu/praglm/
📅 Submit by June 23rd
How do LLMs engage in pragmatic reasoning, and what core pragmatic capacities remain beyond their reach?
🌐 sites.google.com/berkeley.edu/praglm/
📅 Submit by June 23rd
PragLM @ COLM '25
IMPORTANT DATES
sites.google.com
May 28, 2025 at 6:21 PM
Happy to announce the first workshop on Pragmatic Reasoning in Language Models — PragLM @ COLM 2025! 🎉
How do LLMs engage in pragmatic reasoning, and what core pragmatic capacities remain beyond their reach?
🌐 sites.google.com/berkeley.edu/praglm/
📅 Submit by June 23rd
How do LLMs engage in pragmatic reasoning, and what core pragmatic capacities remain beyond their reach?
🌐 sites.google.com/berkeley.edu/praglm/
📅 Submit by June 23rd
Reposted by Daniel Fried
I'm joining Wisconsin CS as an assistant professor in fall 2026!! There, I'll continue working on language models, computational social science, & responsible AI. 🌲🧀🚣🏻♀️ Apply to be my PhD student!
Before then, I'll postdoc for a year in the NLP group at another UW 🏔️ in the Pacific Northwest
Before then, I'll postdoc for a year in the NLP group at another UW 🏔️ in the Pacific Northwest
May 5, 2025 at 7:54 PM
I'm joining Wisconsin CS as an assistant professor in fall 2026!! There, I'll continue working on language models, computational social science, & responsible AI. 🌲🧀🚣🏻♀️ Apply to be my PhD student!
Before then, I'll postdoc for a year in the NLP group at another UW 🏔️ in the Pacific Northwest
Before then, I'll postdoc for a year in the NLP group at another UW 🏔️ in the Pacific Northwest
Reposted by Daniel Fried
Inaugurating new acct to share work from my PhD student!
Wayne et al have been running a live eval platform Copilot Arena - a VSCode extension serving code completions from AI systems to real developers. See 🧵 for findings and preprint
Excited to be evaluating human-AI *workflows* holistically!
Wayne et al have been running a live eval platform Copilot Arena - a VSCode extension serving code completions from AI systems to real developers. See 🧵 for findings and preprint
Excited to be evaluating human-AI *workflows* holistically!
What do developers 𝘳𝘦𝘢𝘭𝘭𝘺 think of AI coding assistants?
In October, we launched Copilot Arena to collect user preferences on real dev workflows. After months of live service, we’re here to share our findings in our recent preprint.
Here's what we have learned /🧵
In October, we launched Copilot Arena to collect user preferences on real dev workflows. After months of live service, we’re here to share our findings in our recent preprint.
Here's what we have learned /🧵
March 5, 2025 at 5:01 PM
Inaugurating new acct to share work from my PhD student!
Wayne et al have been running a live eval platform Copilot Arena - a VSCode extension serving code completions from AI systems to real developers. See 🧵 for findings and preprint
Excited to be evaluating human-AI *workflows* holistically!
Wayne et al have been running a live eval platform Copilot Arena - a VSCode extension serving code completions from AI systems to real developers. See 🧵 for findings and preprint
Excited to be evaluating human-AI *workflows* holistically!
Reposted by Daniel Fried
What if AI agents did software engineering like humans—seeing the screen & using any developer tool?
Introducing Programming with Pixels: an SWE environment where agents control VSCode via screen perception, typing & clicking to tackle diverse tasks.
programmingwithpixels.com
🧵
Introducing Programming with Pixels: an SWE environment where agents control VSCode via screen perception, typing & clicking to tackle diverse tasks.
programmingwithpixels.com
🧵
February 26, 2025 at 5:17 PM
What if AI agents did software engineering like humans—seeing the screen & using any developer tool?
Introducing Programming with Pixels: an SWE environment where agents control VSCode via screen perception, typing & clicking to tackle diverse tasks.
programmingwithpixels.com
🧵
Introducing Programming with Pixels: an SWE environment where agents control VSCode via screen perception, typing & clicking to tackle diverse tasks.
programmingwithpixels.com
🧵
Reposted by Daniel Fried
Interested in knowing more about LLMs agents and in contributing to this topic?🚀
📢We're thrilled to announce REALM: The first Workshop for Research on Agent Language Models 🤖 #ACL2025NLP in Vienna 🎻
We have an exciting lineup of speakers
🗓️ Submit your work by *March 1st*
@aclmeeting.bsky.social
📢We're thrilled to announce REALM: The first Workshop for Research on Agent Language Models 🤖 #ACL2025NLP in Vienna 🎻
We have an exciting lineup of speakers
🗓️ Submit your work by *March 1st*
@aclmeeting.bsky.social
January 23, 2025 at 2:29 PM
Interested in knowing more about LLMs agents and in contributing to this topic?🚀
📢We're thrilled to announce REALM: The first Workshop for Research on Agent Language Models 🤖 #ACL2025NLP in Vienna 🎻
We have an exciting lineup of speakers
🗓️ Submit your work by *March 1st*
@aclmeeting.bsky.social
📢We're thrilled to announce REALM: The first Workshop for Research on Agent Language Models 🤖 #ACL2025NLP in Vienna 🎻
We have an exciting lineup of speakers
🗓️ Submit your work by *March 1st*
@aclmeeting.bsky.social
Reposted by Daniel Fried
Thrilled to announce our new work TestGenEval, a benchmark that measures unit test generation and test completion capabilities. This work was done in collaboration with the FAIR CodeGen team.
Preprint: arxiv.org/abs/2410.00752
Leaderboard: testgeneval.github.io/leaderboard....
Preprint: arxiv.org/abs/2410.00752
Leaderboard: testgeneval.github.io/leaderboard....
December 19, 2024 at 8:59 PM
Thrilled to announce our new work TestGenEval, a benchmark that measures unit test generation and test completion capabilities. This work was done in collaboration with the FAIR CodeGen team.
Preprint: arxiv.org/abs/2410.00752
Leaderboard: testgeneval.github.io/leaderboard....
Preprint: arxiv.org/abs/2410.00752
Leaderboard: testgeneval.github.io/leaderboard....
Reposted by Daniel Fried
CMU LTI is hosting predoc interns this summer, centered around "Language Technologies for All"! Please apply and circulate! lti.cs.cmu.edu/news-and-eve...
CMU LTI Language Technology for All Internship 2025 - Language Technologies Institute - School of Computer Science - Carnegie Mellon University
The LTI is currently seeking applicants for the summer 2025 Language Technology for All Internship
lti.cs.cmu.edu
January 7, 2025 at 10:42 PM
CMU LTI is hosting predoc interns this summer, centered around "Language Technologies for All"! Please apply and circulate! lti.cs.cmu.edu/news-and-eve...
Reposted by Daniel Fried
Announcement #1: our call for papers is up! 🎉
colmweb.org/cfp.html
And excited to announce the COLM 2025 program chairs @yoavartzi.com @eunsol.bsky.social @ranjaykrishna.bsky.social and @adtraghunathan.bsky.social
colmweb.org/cfp.html
And excited to announce the COLM 2025 program chairs @yoavartzi.com @eunsol.bsky.social @ranjaykrishna.bsky.social and @adtraghunathan.bsky.social
December 17, 2024 at 3:48 PM
Announcement #1: our call for papers is up! 🎉
colmweb.org/cfp.html
And excited to announce the COLM 2025 program chairs @yoavartzi.com @eunsol.bsky.social @ranjaykrishna.bsky.social and @adtraghunathan.bsky.social
colmweb.org/cfp.html
And excited to announce the COLM 2025 program chairs @yoavartzi.com @eunsol.bsky.social @ranjaykrishna.bsky.social and @adtraghunathan.bsky.social