Mimansa Jaiswal
@mimansaj.bsky.social
Robustness, Data & Annotations, Evaluation & Interpretability in LLMs
http://mimansajaiswal.github.io/
http://mimansajaiswal.github.io/
Reposted by Mimansa Jaiswal
I am disappointed in the AI discourse steveklabnik.com/writing/i-am...
I am disappointed in the AI discourse
steveklabnik.com
May 28, 2025 at 5:33 PM
I am disappointed in the AI discourse steveklabnik.com/writing/i-am...
Reposted by Mimansa Jaiswal
Meta introduced Llama 4 models and added this section near the very bottom of the announcement 😬
“[LLMs] historically have leaned left when it comes to debated political and social topics.”
ai.meta.com/blog/llama-4...
“[LLMs] historically have leaned left when it comes to debated political and social topics.”
ai.meta.com/blog/llama-4...
April 5, 2025 at 10:08 PM
Meta introduced Llama 4 models and added this section near the very bottom of the announcement 😬
“[LLMs] historically have leaned left when it comes to debated political and social topics.”
ai.meta.com/blog/llama-4...
“[LLMs] historically have leaned left when it comes to debated political and social topics.”
ai.meta.com/blog/llama-4...
Reposted by Mimansa Jaiswal
"We train our LLMs on art and literature and educational materials, and for some reason they keep turning out progressive."
April 5, 2025 at 11:33 PM
"We train our LLMs on art and literature and educational materials, and for some reason they keep turning out progressive."
Reposted by Mimansa Jaiswal
Have work on the actionable impact of interpretability findings? Consider submitting to our Actionable Interpretability workshop at ICML! See below for more info.
Website: actionable-interpretability.github.io
Deadline: May 9
Website: actionable-interpretability.github.io
Deadline: May 9
🎉 Our Actionable Interpretability workshop has been accepted to #ICML2025! 🎉
> Follow @actinterp.bsky.social
> Website actionable-interpretability.github.io
@talhaklay.bsky.social @anja.re @mariusmosbach.bsky.social @sarah-nlp.bsky.social @iftenney.bsky.social
Paper submission deadline: May 9th!
> Follow @actinterp.bsky.social
> Website actionable-interpretability.github.io
@talhaklay.bsky.social @anja.re @mariusmosbach.bsky.social @sarah-nlp.bsky.social @iftenney.bsky.social
Paper submission deadline: May 9th!
April 3, 2025 at 5:58 PM
Have work on the actionable impact of interpretability findings? Consider submitting to our Actionable Interpretability workshop at ICML! See below for more info.
Website: actionable-interpretability.github.io
Deadline: May 9
Website: actionable-interpretability.github.io
Deadline: May 9
Reposted by Mimansa Jaiswal
𝐇𝐨𝐰 𝐜𝐚𝐧 𝐰𝐞 𝐩𝐞𝐫𝐟𝐞𝐜𝐭𝐥𝐲 𝐞𝐫𝐚𝐬𝐞 𝐜𝐨𝐧𝐜𝐞𝐩𝐭𝐬 𝐟𝐫𝐨𝐦 𝐋𝐋𝐌𝐬?
Our method, Perfect Erasure Functions (PEF), erases concepts perfectly from LLM representations. We analytically derive PEF w/o parameter estimation. PEFs achieve pareto optimal erasure-utility tradeoff backed w/ theoretical guarantees. #AISTATS2025 🧵
Our method, Perfect Erasure Functions (PEF), erases concepts perfectly from LLM representations. We analytically derive PEF w/o parameter estimation. PEFs achieve pareto optimal erasure-utility tradeoff backed w/ theoretical guarantees. #AISTATS2025 🧵
April 2, 2025 at 4:03 PM
𝐇𝐨𝐰 𝐜𝐚𝐧 𝐰𝐞 𝐩𝐞𝐫𝐟𝐞𝐜𝐭𝐥𝐲 𝐞𝐫𝐚𝐬𝐞 𝐜𝐨𝐧𝐜𝐞𝐩𝐭𝐬 𝐟𝐫𝐨𝐦 𝐋𝐋𝐌𝐬?
Our method, Perfect Erasure Functions (PEF), erases concepts perfectly from LLM representations. We analytically derive PEF w/o parameter estimation. PEFs achieve pareto optimal erasure-utility tradeoff backed w/ theoretical guarantees. #AISTATS2025 🧵
Our method, Perfect Erasure Functions (PEF), erases concepts perfectly from LLM representations. We analytically derive PEF w/o parameter estimation. PEFs achieve pareto optimal erasure-utility tradeoff backed w/ theoretical guarantees. #AISTATS2025 🧵
Reposted by Mimansa Jaiswal
New paper from our team @GoogleDeepMind!
🚨 We've put LLMs to the test as writing co-pilots – how good are they really at helping us write? LLMs are increasingly used for open-ended tasks like writing assistance, but how do we assess their effectiveness? 🤔
arxiv.org/pdf/2503.19711
🚨 We've put LLMs to the test as writing co-pilots – how good are they really at helping us write? LLMs are increasingly used for open-ended tasks like writing assistance, but how do we assess their effectiveness? 🤔
arxiv.org/pdf/2503.19711
arxiv.org
April 2, 2025 at 9:51 AM
New paper from our team @GoogleDeepMind!
🚨 We've put LLMs to the test as writing co-pilots – how good are they really at helping us write? LLMs are increasingly used for open-ended tasks like writing assistance, but how do we assess their effectiveness? 🤔
arxiv.org/pdf/2503.19711
🚨 We've put LLMs to the test as writing co-pilots – how good are they really at helping us write? LLMs are increasingly used for open-ended tasks like writing assistance, but how do we assess their effectiveness? 🤔
arxiv.org/pdf/2503.19711
Reposted by Mimansa Jaiswal
pre aca you would specifically avoid being diagnosed or seeking treatment if you didn't have health insurance to prevent it from making it impossible for you to get health insurance. when you bought health insurance after doing this you committed fraud.
i did this.
i did this.
March 23, 2025 at 6:38 PM
pre aca you would specifically avoid being diagnosed or seeking treatment if you didn't have health insurance to prevent it from making it impossible for you to get health insurance. when you bought health insurance after doing this you committed fraud.
i did this.
i did this.
Reposted by Mimansa Jaiswal
Some of his readers have asked Mike Masnick @mmasnick.bsky.social why his technology news site, Tech Dirt, has been covering politics so intensely lately. www.techdirt.com/2025/03/04/w...
I cannot recommend Mike's reply enough. It's exactly what readers need to hear, what journalists need to do.
I cannot recommend Mike's reply enough. It's exactly what readers need to hear, what journalists need to do.
March 7, 2025 at 12:09 AM
Some of his readers have asked Mike Masnick @mmasnick.bsky.social why his technology news site, Tech Dirt, has been covering politics so intensely lately. www.techdirt.com/2025/03/04/w...
I cannot recommend Mike's reply enough. It's exactly what readers need to hear, what journalists need to do.
I cannot recommend Mike's reply enough. It's exactly what readers need to hear, what journalists need to do.
Reposted by Mimansa Jaiswal
Neat visualization that came up in the ARBOR project: this shows DeepSeek "thinking" about a question, and color is the probability that, if it exited thinking, it would give the right answer. (Here yellow means correct.)
February 25, 2025 at 6:44 PM
Neat visualization that came up in the ARBOR project: this shows DeepSeek "thinking" about a question, and color is the probability that, if it exited thinking, it would give the right answer. (Here yellow means correct.)
Reposted by Mimansa Jaiswal
Come work with me!
We are looking to bring on more top talent to our language modeling workstream at @ai2.bsky.social building the open ecosystem. We are hiring:
* Research scientists
* Senior research engineers
* Post docs (Young investigators)
* Pre docs
job-boards.greenhouse.io/thealleninst...
We are looking to bring on more top talent to our language modeling workstream at @ai2.bsky.social building the open ecosystem. We are hiring:
* Research scientists
* Senior research engineers
* Post docs (Young investigators)
* Pre docs
job-boards.greenhouse.io/thealleninst...
The Allen Institute for AI
job-boards.greenhouse.io
February 25, 2025 at 1:07 AM
Come work with me!
We are looking to bring on more top talent to our language modeling workstream at @ai2.bsky.social building the open ecosystem. We are hiring:
* Research scientists
* Senior research engineers
* Post docs (Young investigators)
* Pre docs
job-boards.greenhouse.io/thealleninst...
We are looking to bring on more top talent to our language modeling workstream at @ai2.bsky.social building the open ecosystem. We are hiring:
* Research scientists
* Senior research engineers
* Post docs (Young investigators)
* Pre docs
job-boards.greenhouse.io/thealleninst...
I interviewed for LLM/ML research scientist/engineering positions last Fall. Over 200 applications, 100 interviews, many rejections & some offers later, I decided to write the process down, along with the resources I used.
Links to the process & resources in the following tweets
Links to the process & resources in the following tweets
February 24, 2025 at 5:24 PM
I interviewed for LLM/ML research scientist/engineering positions last Fall. Over 200 applications, 100 interviews, many rejections & some offers later, I decided to write the process down, along with the resources I used.
Links to the process & resources in the following tweets
Links to the process & resources in the following tweets
Reposted by Mimansa Jaiswal
Obsessed with the work coming out of Finale Doshi-Velez's group; they don't just take the limits of the real world for ML deployment seriously but instead turn it into new algorithmic ideas
arxiv.org/abs/2406.08636
arxiv.org/abs/2406.08636
Towards Integrating Personal Knowledge into Test-Time Predictions
Machine learning (ML) models can make decisions based on large amounts of data, but they can be missing personal knowledge available to human users about whom predictions are made. For example, a mode...
arxiv.org
February 13, 2025 at 4:13 AM
Obsessed with the work coming out of Finale Doshi-Velez's group; they don't just take the limits of the real world for ML deployment seriously but instead turn it into new algorithmic ideas
arxiv.org/abs/2406.08636
arxiv.org/abs/2406.08636
Reposted by Mimansa Jaiswal
The entire archive of CDC datasets can be found here.
HUGE shoutout to data archivists- this work is important 👏🙌🏻
archive.org/details/2025...
HUGE shoutout to data archivists- this work is important 👏🙌🏻
archive.org/details/2025...
February 1, 2025 at 6:33 PM
The entire archive of CDC datasets can be found here.
HUGE shoutout to data archivists- this work is important 👏🙌🏻
archive.org/details/2025...
HUGE shoutout to data archivists- this work is important 👏🙌🏻
archive.org/details/2025...
Reposted by Mimansa Jaiswal
Can AI really help with literature reviews? 🧐
Meet Ai2 ScholarQA, an experimental solution that allows you to ask questions that require multiple scientific papers to answer. It gives more in-depth and contextual answers with table comparisons and expandable sections 💡
Try it now: scholarqa.allen.ai
Meet Ai2 ScholarQA, an experimental solution that allows you to ask questions that require multiple scientific papers to answer. It gives more in-depth and contextual answers with table comparisons and expandable sections 💡
Try it now: scholarqa.allen.ai
January 21, 2025 at 7:31 PM
Can AI really help with literature reviews? 🧐
Meet Ai2 ScholarQA, an experimental solution that allows you to ask questions that require multiple scientific papers to answer. It gives more in-depth and contextual answers with table comparisons and expandable sections 💡
Try it now: scholarqa.allen.ai
Meet Ai2 ScholarQA, an experimental solution that allows you to ask questions that require multiple scientific papers to answer. It gives more in-depth and contextual answers with table comparisons and expandable sections 💡
Try it now: scholarqa.allen.ai
Reposted by Mimansa Jaiswal
It is such a slap in the face to the Indian American community to delay their green cards for decades and then declare that because of that delay their American children aren't citizens.
January 21, 2025 at 3:30 AM
It is such a slap in the face to the Indian American community to delay their green cards for decades and then declare that because of that delay their American children aren't citizens.
Is it time for a social media break again? It has not been a great day. 😅
This probably won’t reach many people, but if someone who feels the same way ends up reading this, I hope it helps them realize they’re not alone.
Lately, my mood has been heavily influenced by the ‘tech(adjacent) twitter' vibes, & the past few months have been really rough.
⏎
Lately, my mood has been heavily influenced by the ‘tech(adjacent) twitter' vibes, & the past few months have been really rough.
⏎
January 21, 2025 at 7:32 AM
Is it time for a social media break again? It has not been a great day. 😅
This is pretty cool!
Learn more: developer.chrome.com/docs/devtool... (seems to cover CSS and network requests, which might be fun to lay around with)
Learn more: developer.chrome.com/docs/devtool... (seems to cover CSS and network requests, which might be fun to lay around with)
January 6, 2025 at 8:25 PM
This is pretty cool!
Learn more: developer.chrome.com/docs/devtool... (seems to cover CSS and network requests, which might be fun to lay around with)
Learn more: developer.chrome.com/docs/devtool... (seems to cover CSS and network requests, which might be fun to lay around with)
Reposted by Mimansa Jaiswal
i was annoyed at having many chrome tabs with PDF papers having uninformative titles, so i created a small chrome extension to fix it.
i'm using it for a while now, works well.
today i put it on github. enjoy.
github.com/yoavg/pdf-ta...
i'm using it for a while now, works well.
today i put it on github. enjoy.
github.com/yoavg/pdf-ta...
January 5, 2025 at 10:22 PM
i was annoyed at having many chrome tabs with PDF papers having uninformative titles, so i created a small chrome extension to fix it.
i'm using it for a while now, works well.
today i put it on github. enjoy.
github.com/yoavg/pdf-ta...
i'm using it for a while now, works well.
today i put it on github. enjoy.
github.com/yoavg/pdf-ta...
Reposted by Mimansa Jaiswal
It's ready! 💫
A new blog post in which I list of all the tools and apps I've been using for work, plus all my opinions about them.
maria-antoniak.github.io/2024/12/30/o...
Featuring @kagi.com, @warp.dev, @paperpile.bsky.social, @are.na, Fantastical, @obsidian.md, Claude, and more.
A new blog post in which I list of all the tools and apps I've been using for work, plus all my opinions about them.
maria-antoniak.github.io/2024/12/30/o...
Featuring @kagi.com, @warp.dev, @paperpile.bsky.social, @are.na, Fantastical, @obsidian.md, Claude, and more.
So far the blog post draft is winning the distraction battle. Prepare for a very long and opinionated update about all the new tools and apps I’ve been using for work.
Flight prep for someone who hates flying:
- Switch with Nine Sols loaded
- iPad with Black Doves loaded
- laptop with data, python notebook, blog post draft loaded
- silk eye mask
- REI inflatable neck pillow
- vitamin C juice
- Journey to the East by Hermann Hesse
- compression socks
- many snacks
- Switch with Nine Sols loaded
- iPad with Black Doves loaded
- laptop with data, python notebook, blog post draft loaded
- silk eye mask
- REI inflatable neck pillow
- vitamin C juice
- Journey to the East by Hermann Hesse
- compression socks
- many snacks
December 31, 2024 at 5:38 AM
It's ready! 💫
A new blog post in which I list of all the tools and apps I've been using for work, plus all my opinions about them.
maria-antoniak.github.io/2024/12/30/o...
Featuring @kagi.com, @warp.dev, @paperpile.bsky.social, @are.na, Fantastical, @obsidian.md, Claude, and more.
A new blog post in which I list of all the tools and apps I've been using for work, plus all my opinions about them.
maria-antoniak.github.io/2024/12/30/o...
Featuring @kagi.com, @warp.dev, @paperpile.bsky.social, @are.na, Fantastical, @obsidian.md, Claude, and more.
Reposted by Mimansa Jaiswal
I've always wanted to build things with D3, but the learning curve was too high. At least for the bespoke stuff I wanted to make (not just simple bar charts).
I can finally make things like this thanks to Cursor! I just art directed this, and it made everything work beautifully. Even on mobile 🎉
I can finally make things like this thanks to Cursor! I just art directed this, and it made everything work beautifully. Even on mobile 🎉
December 31, 2024 at 5:18 PM
I've always wanted to build things with D3, but the learning curve was too high. At least for the bespoke stuff I wanted to make (not just simple bar charts).
I can finally make things like this thanks to Cursor! I just art directed this, and it made everything work beautifully. Even on mobile 🎉
I can finally make things like this thanks to Cursor! I just art directed this, and it made everything work beautifully. Even on mobile 🎉
If you like working with a canvas, Muse (museapp.com) currently has a 30% off. 2 major things that make it different than freeform -- ink sticks to sticky notes (so it feels more like writing in the real world), and you can snippet out sections from pdf that link back to source.
Inspired & focused thinking with Muse
Muse is a canvas for thinking that helps you get clarity on things that matter. Think in private or collaborate with others. Available for iPad and Mac.
museapp.com
December 29, 2024 at 11:55 PM
If you like working with a canvas, Muse (museapp.com) currently has a 30% off. 2 major things that make it different than freeform -- ink sticks to sticky notes (so it feels more like writing in the real world), and you can snippet out sections from pdf that link back to source.
I don't usually discuss software, PKM, or tools here, but I found a valuable tip today.
Not only can you use this to create beautiful video tutorials similar to what Screenstudio creates automatically, but you can also use this feature during live sharing & streaming, making it incredibly useful!
Not only can you use this to create beautiful video tutorials similar to what Screenstudio creates automatically, but you can also use this feature during live sharing & streaming, making it incredibly useful!
🔥 One of my favourite hidden macOS features is the scroll hotkey gesture. By holding ⌘ (Command) and scrolling down, we zoom WAY IN on the cursor’s location.
I learned this trick for highlighting stuff in video tutorials, but it comes in handy a lot in my day-to-day life!
I learned this trick for highlighting stuff in video tutorials, but it comes in handy a lot in my day-to-day life!
December 26, 2024 at 9:15 PM
I don't usually discuss software, PKM, or tools here, but I found a valuable tip today.
Not only can you use this to create beautiful video tutorials similar to what Screenstudio creates automatically, but you can also use this feature during live sharing & streaming, making it incredibly useful!
Not only can you use this to create beautiful video tutorials similar to what Screenstudio creates automatically, but you can also use this feature during live sharing & streaming, making it incredibly useful!
This probably won’t reach many people, but if someone who feels the same way ends up reading this, I hope it helps them realize they’re not alone.
Lately, my mood has been heavily influenced by the ‘tech(adjacent) twitter' vibes, & the past few months have been really rough.
⏎
Lately, my mood has been heavily influenced by the ‘tech(adjacent) twitter' vibes, & the past few months have been really rough.
⏎
December 26, 2024 at 2:29 PM
This probably won’t reach many people, but if someone who feels the same way ends up reading this, I hope it helps them realize they’re not alone.
Lately, my mood has been heavily influenced by the ‘tech(adjacent) twitter' vibes, & the past few months have been really rough.
⏎
Lately, my mood has been heavily influenced by the ‘tech(adjacent) twitter' vibes, & the past few months have been really rough.
⏎
Reposted by Mimansa Jaiswal
The other major Chinese AI lab, DeepSeek, just dropped their own last-minute entry into the 2024 model race: DeepSeek v3 is a HUGE model (685B parameters) which showed up, mostly undocumented, on Hugging Face this morning. My notes so far: https://simonwillison.net/2024/Dec/25/deepseek-v3/
deepseek-ai/DeepSeek-V3-Base
No model card or announcement yet, but this new model release from Chinese AI lab DeepSeek (an arm of Chinese hedge fund [High-Flyer](https://en.wikipedia.org/wiki/High-Flyer_(company))) looks very significant. It's a huge model …
simonwillison.net
December 25, 2024 at 7:03 PM
The other major Chinese AI lab, DeepSeek, just dropped their own last-minute entry into the 2024 model race: DeepSeek v3 is a HUGE model (685B parameters) which showed up, mostly undocumented, on Hugging Face this morning. My notes so far: https://simonwillison.net/2024/Dec/25/deepseek-v3/