Shahan Ali Memon
@shahanmemon.bsky.social
Researching {science of AI-mediated science, metascience #SciSci, #AI4Science, generative #AI, LLMs, agents, alignment, misinformation in science}
PhD @ UW.
Visiting @ NYU & MSR
Alum @ Carnegie Mellon
Academic webpage: https://samemon.github.io
PhD @ UW.
Visiting @ NYU & MSR
Alum @ Carnegie Mellon
Academic webpage: https://samemon.github.io
"Finally, after five months of dating, Kasper decided to propose! ... Kasper described what kind of ring he would like to give me (blue is my favorite color...), I found a few online that I liked, sent him photos and he chose the one... I love him more than anything in the world and I am so happy!"
November 16, 2025 at 5:23 AM
"Finally, after five months of dating, Kasper decided to propose! ... Kasper described what kind of ring he would like to give me (blue is my favorite color...), I found a few online that I liked, sent him photos and he chose the one... I love him more than anything in the world and I am so happy!"
Some more context..
November 14, 2025 at 1:01 PM
Some more context..
On the other end of the spectrum, we have reviews like this:
November 14, 2025 at 12:51 PM
On the other end of the spectrum, we have reviews like this:
Gave it to GPT 5.1 just to test and it gave a score of 8.
November 14, 2025 at 12:42 PM
Gave it to GPT 5.1 just to test and it gave a score of 8.
“Can ChatGPT/AI do X” in titles?
November 14, 2025 at 10:45 AM
“Can ChatGPT/AI do X” in titles?
But in science too. This phrase seems to have undergone semantic satiation because of its overuse. Are there other cliché phrases?
November 14, 2025 at 10:45 AM
But in science too. This phrase seems to have undergone semantic satiation because of its overuse. Are there other cliché phrases?
October 20, 2025 at 9:06 PM
The future of peer review is here.
Introducing Nature Instant!
We've automated rejection so you can get disappointed faster. Because why wait 6 months for reviewer 2 when you can get roasted by an #AI in seconds?
Here is how it works.
#AI4Science #PeerReview #FictionScience #FiSci #AAAI #ScAISci
Introducing Nature Instant!
We've automated rejection so you can get disappointed faster. Because why wait 6 months for reviewer 2 when you can get roasted by an #AI in seconds?
Here is how it works.
#AI4Science #PeerReview #FictionScience #FiSci #AAAI #ScAISci
October 5, 2025 at 6:17 AM
The future of peer review is here.
Introducing Nature Instant!
We've automated rejection so you can get disappointed faster. Because why wait 6 months for reviewer 2 when you can get roasted by an #AI in seconds?
Here is how it works.
#AI4Science #PeerReview #FictionScience #FiSci #AAAI #ScAISci
Introducing Nature Instant!
We've automated rejection so you can get disappointed faster. Because why wait 6 months for reviewer 2 when you can get roasted by an #AI in seconds?
Here is how it works.
#AI4Science #PeerReview #FictionScience #FiSci #AAAI #ScAISci
Applying for a #PhD @ischool.uw.edu? Read 👇
Our student-run application feedback program will be open from October 20th through 1st November 2025.
Everyone applying, especially those from historically underrepresented groups or who have faced barriers in higher ed are highly encouraged to apply.
Our student-run application feedback program will be open from October 20th through 1st November 2025.
Everyone applying, especially those from historically underrepresented groups or who have faced barriers in higher ed are highly encouraged to apply.
September 26, 2025 at 11:00 PM
Applying for a #PhD @ischool.uw.edu? Read 👇
Our student-run application feedback program will be open from October 20th through 1st November 2025.
Everyone applying, especially those from historically underrepresented groups or who have faced barriers in higher ed are highly encouraged to apply.
Our student-run application feedback program will be open from October 20th through 1st November 2025.
Everyone applying, especially those from historically underrepresented groups or who have faced barriers in higher ed are highly encouraged to apply.
Mandatory screenshot save as the “instructor on record” 🥳
September 25, 2025 at 4:26 AM
Mandatory screenshot save as the “instructor on record” 🥳
I just taught my first lecture as a predoctoral instructor for “INFO 371: Advanced Methods in Data Science” at the University of Washington Information School—and it was an incredible experience.
#academia
#academia
September 25, 2025 at 4:18 AM
I just taught my first lecture as a predoctoral instructor for “INFO 371: Advanced Methods in Data Science” at the University of Washington Information School—and it was an incredible experience.
#academia
#academia
I am not an AI pessimist, but we must think carefully where it belongs, and where it does not. Good AI systems should subtract human agency in the right places.
14/15
14/15
September 14, 2025 at 9:12 PM
I am not an AI pessimist, but we must think carefully where it belongs, and where it does not. Good AI systems should subtract human agency in the right places.
14/15
14/15
What exactly is the end goal of an autonomous "AI scientist"? Is it to build a perpetual discovery machine, an autonomous system that tirelessly produces new knowledge while we sit comfortably on our sofas, enjoying the perks of life?
2/15
2/15
September 14, 2025 at 9:12 PM
What exactly is the end goal of an autonomous "AI scientist"? Is it to build a perpetual discovery machine, an autonomous system that tirelessly produces new knowledge while we sit comfortably on our sofas, enjoying the perks of life?
2/15
2/15
We standardize 5.2 million job titles into 123 thousand unique titles and propose a Specialization Index to quantify how concentrated a given title is within a sector.
July 14, 2025 at 5:10 PM
We standardize 5.2 million job titles into 123 thousand unique titles and propose a Specialization Index to quantify how concentrated a given title is within a sector.
Our new paper mapping job titles, sector specialization, and promotions across 24 sectors now published in @natureportfolio.nature.com Scientific Data
in collaboration w/ Shehryar Subhani and Bedoor Alshebli
@ischool.uw.edu @nyuad-oaa.bsky.social
#LaborEconomics #CareerData #CareerMobility
in collaboration w/ Shehryar Subhani and Bedoor Alshebli
@ischool.uw.edu @nyuad-oaa.bsky.social
#LaborEconomics #CareerData #CareerMobility
July 14, 2025 at 5:10 PM
Our new paper mapping job titles, sector specialization, and promotions across 24 sectors now published in @natureportfolio.nature.com Scientific Data
in collaboration w/ Shehryar Subhani and Bedoor Alshebli
@ischool.uw.edu @nyuad-oaa.bsky.social
#LaborEconomics #CareerData #CareerMobility
in collaboration w/ Shehryar Subhani and Bedoor Alshebli
@ischool.uw.edu @nyuad-oaa.bsky.social
#LaborEconomics #CareerData #CareerMobility
Shoot. Now that I think about it, this would have been such a better representation. I'll do better next time!!
June 30, 2025 at 6:36 AM
Shoot. Now that I think about it, this would have been such a better representation. I'll do better next time!!
Science can be about looking outward...or inward. Both are valid. And context matters. Many times, you need introspection to grow, evolve, and do better work.
Not all mirrors are distractions. Some are necessary.
🧵 12/12
Not all mirrors are distractions. Some are necessary.
🧵 12/12
June 30, 2025 at 6:18 AM
Science can be about looking outward...or inward. Both are valid. And context matters. Many times, you need introspection to grow, evolve, and do better work.
Not all mirrors are distractions. Some are necessary.
🧵 12/12
Not all mirrors are distractions. Some are necessary.
🧵 12/12
On "navel-gazing" in science..
and why we should rethink how we use this term in research discourse.
🧵 1/12
#SciSci #MetaScience #MetaSci #ResearchOnResearch #RoR
and why we should rethink how we use this term in research discourse.
🧵 1/12
#SciSci #MetaScience #MetaSci #ResearchOnResearch #RoR
June 30, 2025 at 6:18 AM
On "navel-gazing" in science..
and why we should rethink how we use this term in research discourse.
🧵 1/12
#SciSci #MetaScience #MetaSci #ResearchOnResearch #RoR
and why we should rethink how we use this term in research discourse.
🧵 1/12
#SciSci #MetaScience #MetaSci #ResearchOnResearch #RoR
All new submissions to @nathumbehav.nature.com that are published will be accompanied by referees’ reports and author responses. Let’s go!!!!
Anyone (wanna) study(ing) this as a natural experiment? 🙃
#Metascience #SciSci #TransparentScience
Anyone (wanna) study(ing) this as a natural experiment? 🙃
#Metascience #SciSci #TransparentScience
June 19, 2025 at 7:36 AM
All new submissions to @nathumbehav.nature.com that are published will be accompanied by referees’ reports and author responses. Let’s go!!!!
Anyone (wanna) study(ing) this as a natural experiment? 🙃
#Metascience #SciSci #TransparentScience
Anyone (wanna) study(ing) this as a natural experiment? 🙃
#Metascience #SciSci #TransparentScience
June 19, 2025 at 6:11 AM
That said, the evidence seems mixed. See for example this: arxiv.org/pdf/2305.13534 and this: arxiv.org/abs/2505.236... and this: arxiv.org/abs/2505.21523
According to system card, o3 and o4 seem to hallucinate much more than o1 🤷♂️
According to system card, o3 and o4 seem to hallucinate much more than o1 🤷♂️
June 5, 2025 at 7:47 AM
That said, the evidence seems mixed. See for example this: arxiv.org/pdf/2305.13534 and this: arxiv.org/abs/2505.236... and this: arxiv.org/abs/2505.21523
According to system card, o3 and o4 seem to hallucinate much more than o1 🤷♂️
According to system card, o3 and o4 seem to hallucinate much more than o1 🤷♂️
Actually, it's a bit different than an LLM model. These screenshots are from a thread on twitter than I have found useful in understanding DeepResearch + its failure modes.
June 4, 2025 at 7:14 AM
Actually, it's a bit different than an LLM model. These screenshots are from a thread on twitter than I have found useful in understanding DeepResearch + its failure modes.
