Danielle Bitterman MD
@daniellebitterman.bsky.social
I'm a physician-scientist working in clinical NLP and LLM safety/evaluation. You'll find me in the lab or the rad onc clinic | BWH | DFCI | Harvard Medical School
www.bittermanlab.org
www.bittermanlab.org
Pinned
What We Learned About LLM/VLMs in Healthcare AI Evaluation:
A Blog post by Shan Chen on Hugging Face
huggingface.co
🩺💡The Bitterman lab has spent much of the past year researching #LLMs for healthcare. This post summarizes our inroads into making LLMs safer and reliable for clinicians and patients: huggingface.co/blog/shanche....
We'll be at #EMNLP2024 - come chat if you have similar interests!
We'll be at #EMNLP2024 - come chat if you have similar interests!
Super proud of @shan23chen.bsky.social for his podium presentation on his research into LLM sycophancy in the face of illogical medical queries at #AMIA25!
Full paper: www.nature.com/articles/s41...
Also cited yesterday in the NYT! www.nytimes.com/2025/11/16/w...
Full paper: www.nature.com/articles/s41...
Also cited yesterday in the NYT! www.nytimes.com/2025/11/16/w...
November 17, 2025 at 9:44 PM
Super proud of @shan23chen.bsky.social for his podium presentation on his research into LLM sycophancy in the face of illogical medical queries at #AMIA25!
Full paper: www.nature.com/articles/s41...
Also cited yesterday in the NYT! www.nytimes.com/2025/11/16/w...
Full paper: www.nature.com/articles/s41...
Also cited yesterday in the NYT! www.nytimes.com/2025/11/16/w...
LLMs tend to prioritize helpfulness > reason. We show that safety-aware, compute-efficient fine-tuning helps models reason more critically in healthcare domain, and generalizes to improved safety alignment across other domains.
www.nature.com/articles/s41... @shan23chen.bsky.social
www.nature.com/articles/s41... @shan23chen.bsky.social
When helpfulness backfires: LLMs and the risk of false medical information due to sycophantic behavior - npj Digital Medicine
npj Digital Medicine - When helpfulness backfires: LLMs and the risk of false medical information due to sycophantic behavior
www.nature.com
October 18, 2025 at 2:18 PM
LLMs tend to prioritize helpfulness > reason. We show that safety-aware, compute-efficient fine-tuning helps models reason more critically in healthcare domain, and generalizes to improved safety alignment across other domains.
www.nature.com/articles/s41... @shan23chen.bsky.social
www.nature.com/articles/s41... @shan23chen.bsky.social
Reposted by Danielle Bitterman MD
An overemphasis on helpfulness makes LLMs vulnerable.
Research shows models will comply with illogical medical requests, generating false information. This sycophantic tendency can be corrected with specific prompting and fine-tuning. #MedSky #MedAI #MLSky
Research shows models will comply with illogical medical requests, generating false information. This sycophantic tendency can be corrected with specific prompting and fine-tuning. #MedSky #MedAI #MLSky
When helpfulness backfires: LLMs and the risk of false medical information due to sycophantic behavior - npj Digital Medicine
npj Digital Medicine - When helpfulness backfires: LLMs and the risk of false medical information due to sycophantic behavior
www.nature.com
October 17, 2025 at 3:53 PM
Reposted by Danielle Bitterman MD
Mass General physician-scientist @daniellebitterman.bsky.social discusses how AI assists the clinical data pipeline leading to better treatments for patients. Listen to unNatural Selection & register for #WMIF2025 at the link in bio to hear more : www.unnaturalselection.net/podcast/s1e19
#MedTech
#MedTech
Clinical Reporting: Mass General — unNatural Selection
Signals Over Noise: Cleaning Up Cancer Trial Data
www.unnaturalselection.net
August 21, 2025 at 4:06 PM
Mass General physician-scientist @daniellebitterman.bsky.social discusses how AI assists the clinical data pipeline leading to better treatments for patients. Listen to unNatural Selection & register for #WMIF2025 at the link in bio to hear more : www.unnaturalselection.net/podcast/s1e19
#MedTech
#MedTech
Reposted by Danielle Bitterman MD
Our paper on multilingual reasoning is accepted to Findings of #EMNLP2025! 🎉 (OA: 3/3/3.5/4)
We show SOTA LMs struggle with reasoning in non-English languages; prompt-hack & post-training improve alignment but trade off accuracy.
📄 arxiv.org/abs/2505.22888
See you in Suzhou! #EMNLP
We show SOTA LMs struggle with reasoning in non-English languages; prompt-hack & post-training improve alignment but trade off accuracy.
📄 arxiv.org/abs/2505.22888
See you in Suzhou! #EMNLP
August 20, 2025 at 8:02 PM
Our paper on multilingual reasoning is accepted to Findings of #EMNLP2025! 🎉 (OA: 3/3/3.5/4)
We show SOTA LMs struggle with reasoning in non-English languages; prompt-hack & post-training improve alignment but trade off accuracy.
📄 arxiv.org/abs/2505.22888
See you in Suzhou! #EMNLP
We show SOTA LMs struggle with reasoning in non-English languages; prompt-hack & post-training improve alignment but trade off accuracy.
📄 arxiv.org/abs/2505.22888
See you in Suzhou! #EMNLP
Are you driven to use AI to transform patient outcomes in oncology? My lab in the AI in Medicine Program (Mass General Brigham, Harvard Medical School) is seeking Postdoctoral Fellows to pioneer applications of AI—especially LLMs—in cancer care. More here: www.linkedin.com/posts/daniel...
🚀 Join Us at the Forefront of AI & Cancer Care | Danielle Bitterman
🚀 Join Us at the Forefront of AI & Cancer Care
Are you driven to use cutting-edge AI to transform patient outcomes in oncology? My lab within the AI in Medicine Program (Mass General Brigham, Har...
www.linkedin.com
July 7, 2025 at 12:22 PM
Are you driven to use AI to transform patient outcomes in oncology? My lab in the AI in Medicine Program (Mass General Brigham, Harvard Medical School) is seeking Postdoctoral Fellows to pioneer applications of AI—especially LLMs—in cancer care. More here: www.linkedin.com/posts/daniel...
Reposted by Danielle Bitterman MD
Reliability of Large Language Model Knowledge Across Brand and Generic Cancer Drug Names | JCO Clinical Cancer Informatics ascopubs.org/doi/abs/10.1... #JCOCCI @daniellebitterman.bsky.social
Reliability of Large Language Model Knowledge Across Brand and Generic Cancer Drug Names | JCO Clinical Cancer Informatics
PURPOSETo evaluate the performance and consistency of large language models (LLMs) across
brand and generic oncology drug names in various clinical tasks, addressing concerns
about potential fluctuati...
ascopubs.org
June 18, 2025 at 5:15 PM
Reliability of Large Language Model Knowledge Across Brand and Generic Cancer Drug Names | JCO Clinical Cancer Informatics ascopubs.org/doi/abs/10.1... #JCOCCI @daniellebitterman.bsky.social
Does your LRM reason in your language? Check out new preprint led by ✨ @jiruiqi.bsky.social & @shan23chen.bsky.social. Implications for safety/human oversight & accuracy!
[1/]💡New Paper
Large reasoning models (LRMs) are strong in English — but how well do they reason in your language?
Our latest work uncovers their limitation and a clear trade-off:
Controlling Thinking Trace Language Comes at the Cost of Accuracy
📄Link: arxiv.org/abs/2505.22888
Large reasoning models (LRMs) are strong in English — but how well do they reason in your language?
Our latest work uncovers their limitation and a clear trade-off:
Controlling Thinking Trace Language Comes at the Cost of Accuracy
📄Link: arxiv.org/abs/2505.22888
May 30, 2025 at 4:25 PM
Does your LRM reason in your language? Check out new preprint led by ✨ @jiruiqi.bsky.social & @shan23chen.bsky.social. Implications for safety/human oversight & accuracy!
Agents are all the rage and we need to track their abilities in the medical domain. Enter MedBrowseComp, the 1st benchmark to assess agents' abilities to reason, navigate the web, and search for verifiable med info!
Preprint: arxiv.org/abs/2505.14963
Site: moreirap12.github.io/mbc-browse-a...
Preprint: arxiv.org/abs/2505.14963
Site: moreirap12.github.io/mbc-browse-a...
May 22, 2025 at 4:27 PM
Agents are all the rage and we need to track their abilities in the medical domain. Enter MedBrowseComp, the 1st benchmark to assess agents' abilities to reason, navigate the web, and search for verifiable med info!
Preprint: arxiv.org/abs/2505.14963
Site: moreirap12.github.io/mbc-browse-a...
Preprint: arxiv.org/abs/2505.14963
Site: moreirap12.github.io/mbc-browse-a...
Reposted by Danielle Bitterman MD
Reposted by Danielle Bitterman MD
I’m thrilled to be in San Francisco for @statnews.com's Breakthrough West Summit! I’ll be bringing my firsthand perspective as a physician-scientist to speak about how AI is transforming cancer care, alongside leaders in the field.
Let's connect if you're here!
#STATBreakthroughSummitWest
Let's connect if you're here!
#STATBreakthroughSummitWest
May 14, 2025 at 1:20 PM
I’m thrilled to be in San Francisco for @statnews.com's Breakthrough West Summit! I’ll be bringing my firsthand perspective as a physician-scientist to speak about how AI is transforming cancer care, alongside leaders in the field.
Let's connect if you're here!
#STATBreakthroughSummitWest
Let's connect if you're here!
#STATBreakthroughSummitWest
Reposted by Danielle Bitterman MD
AI in Cancer Care
Artificial intelligence has the potential to upend oncology, changing everything from diagnosis to treatment options. Get a wide-ranging view of how the use of technology could play out over the next few years.
Moderated by @angusrohan.bsky.social
#STATBreakthrough
Artificial intelligence has the potential to upend oncology, changing everything from diagnosis to treatment options. Get a wide-ranging view of how the use of technology could play out over the next few years.
Moderated by @angusrohan.bsky.social
#STATBreakthrough
May 13, 2025 at 10:59 PM
AI in Cancer Care
Artificial intelligence has the potential to upend oncology, changing everything from diagnosis to treatment options. Get a wide-ranging view of how the use of technology could play out over the next few years.
Moderated by @angusrohan.bsky.social
#STATBreakthrough
Artificial intelligence has the potential to upend oncology, changing everything from diagnosis to treatment options. Get a wide-ranging view of how the use of technology could play out over the next few years.
Moderated by @angusrohan.bsky.social
#STATBreakthrough
Reposted by Danielle Bitterman MD
Exciting news: we are organizing a shared task – 2nd edition of the Chemotherapy Treatment Timelines Extraction from the Clinical Narrative (text mining task) -- collocated with the Clinical NLP Workshop. Do LLMs solve the task? Check out bit.ly/ChemoTimelin...
ChemoTimelines 2025
Treatment regimens are key details in understanding the effects of genetic, epigenetic, and other factors on tumor behavior and responsiveness. As precision oncology progresses, insights into the fine...
bit.ly
April 23, 2025 at 10:59 PM
Exciting news: we are organizing a shared task – 2nd edition of the Chemotherapy Treatment Timelines Extraction from the Clinical Narrative (text mining task) -- collocated with the Clinical NLP Workshop. Do LLMs solve the task? Check out bit.ly/ChemoTimelin...
Reposted by Danielle Bitterman MD
A pie graph worth keeping in mind as the NIH budget plummets jamanetwork.com/journals/jam... for 356 new FDA drugs approved
March 23, 2025 at 4:17 PM
A pie graph worth keeping in mind as the NIH budget plummets jamanetwork.com/journals/jam... for 356 new FDA drugs approved
Conference and professional societies: PLEASE make hybrid options available for attendees and presenters at your conferences so that scientists from HHS-funded agencies can attend. These are unmissable opportunities to promote all the great intramural science and scientists from our government.
February 18, 2025 at 4:34 PM
Conference and professional societies: PLEASE make hybrid options available for attendees and presenters at your conferences so that scientists from HHS-funded agencies can attend. These are unmissable opportunities to promote all the great intramural science and scientists from our government.
Reposted by Danielle Bitterman MD
My Perspective in @NEJM_AI. AI could distort clinical decision-making in ways that prioritize profit over patient care. Oversight & regulation must go beyond performance metrics alone to address hidden commercial forces that could shape decision support. ai.nejm.org/doi/full/10....
Unseen Commercial Forces Could Undermine Artificial Intelligence Decision Support
Artificial intelligence (AI) is poised to transform health care, yet without robust safeguards, unseen commercial interests could distort care by prioritizing profit over patient well-being. The ph...
ai.nejm.org
February 6, 2025 at 4:15 PM
My Perspective in @NEJM_AI. AI could distort clinical decision-making in ways that prioritize profit over patient care. Oversight & regulation must go beyond performance metrics alone to address hidden commercial forces that could shape decision support. ai.nejm.org/doi/full/10....
Reposted by Danielle Bitterman MD
My opinion as an actual NIH-funded researcher (unlike Vinay) at ucsf: his lies about how NIH dollars are used reflect a complete lack of understanding of how research is performed, a lack of respect for research, and are harmful to the entire biomedical research enterprise #grifter
“Dr. Vinay Prasad, a Professor of Epidemiology and Biostatistics and Medicine at the University of California, San Francisco, praised the White House’s NIH's announcement.”
Wonder how any actual researchers at UCSF feel to have a fascist weasel in their ranks selling them out?
Wonder how any actual researchers at UCSF feel to have a fascist weasel in their ranks selling them out?
The White House sent out an email to all reporters on its press list, calling our story “fake news.”
We stand by our story.
We stand by our story.
February 12, 2025 at 2:23 AM
My opinion as an actual NIH-funded researcher (unlike Vinay) at ucsf: his lies about how NIH dollars are used reflect a complete lack of understanding of how research is performed, a lack of respect for research, and are harmful to the entire biomedical research enterprise #grifter
Reposted by Danielle Bitterman MD
Budgeting for the next year of my grants and they will all need to be rescoped, even before the 15% IDC rate. NCI funding at 83% for new awards and another 10% reduction for renewals (current state). Essentially, we are getting 50% of what we asked for...how is this sustainable? @carlbergstrom.com
February 9, 2025 at 3:38 PM
Budgeting for the next year of my grants and they will all need to be rescoped, even before the 15% IDC rate. NCI funding at 83% for new awards and another 10% reduction for renewals (current state). Essentially, we are getting 50% of what we asked for...how is this sustainable? @carlbergstrom.com
As a cancer doctor I see every day how NIH-funded clinical trials save lives and has made the U.S. a leader in medical innovation. Here's one example: In the 1970s, childhood cancer survival was only 58%. Today it is 85%, largely thanks to NIH/NCI funding of Children's Oncology Group trials.
February 5, 2025 at 2:48 PM
As a cancer doctor I see every day how NIH-funded clinical trials save lives and has made the U.S. a leader in medical innovation. Here's one example: In the 1970s, childhood cancer survival was only 58%. Today it is 85%, largely thanks to NIH/NCI funding of Children's Oncology Group trials.
Reposted by Danielle Bitterman MD
Congressional delegation outside USAID now: “We are here to shed a light on a crime unfolding before our eyes.”
February 3, 2025 at 6:00 PM
Congressional delegation outside USAID now: “We are here to shed a light on a crime unfolding before our eyes.”
Reposted by Danielle Bitterman MD
Senator Andy Kim just went to the USAID building, talked to the security guard there to confirm employees are being barred entry, and then did a press gaggle right there in front to call it out.
This is doing something. This is making an effort on messaging. Other Democratic lawmakers: take notes.
This is doing something. This is making an effort on messaging. Other Democratic lawmakers: take notes.
February 3, 2025 at 5:42 PM
Senator Andy Kim just went to the USAID building, talked to the security guard there to confirm employees are being barred entry, and then did a press gaggle right there in front to call it out.
This is doing something. This is making an effort on messaging. Other Democratic lawmakers: take notes.
This is doing something. This is making an effort on messaging. Other Democratic lawmakers: take notes.
Reposted by Danielle Bitterman MD
Gay? Lesbian? Trans? Intersex?
NYC Health has health information for everybody. 🏳️🌈🏳️⚧️
NYC Health has health information for everybody. 🏳️🌈🏳️⚧️
LGBTQ+ Health - NYC Health
www.nyc.gov
February 2, 2025 at 5:43 PM
Gay? Lesbian? Trans? Intersex?
NYC Health has health information for everybody. 🏳️🌈🏳️⚧️
NYC Health has health information for everybody. 🏳️🌈🏳️⚧️
LLM attachment styles: Secure - Claude, Anxious - DeepSeekR1/o1, Avoidant - Gemini
Am I wrong?
Am I wrong?
January 29, 2025 at 10:35 PM
LLM attachment styles: Secure - Claude, Anxious - DeepSeekR1/o1, Avoidant - Gemini
Am I wrong?
Am I wrong?
In addition to the research funded by the NIH, I am grateful and indebted to the dedicated NIH scientists & staff. Their work advances breakthroughs, scientific careers, and improves & saves lives across the U.S
January 29, 2025 at 3:18 AM
In addition to the research funded by the NIH, I am grateful and indebted to the dedicated NIH scientists & staff. Their work advances breakthroughs, scientific careers, and improves & saves lives across the U.S