Scholar

Margaret Mitchell

Margaret Mitchell is a computer scientist who works on algorithmic bias and fairness in machine learning. She is most well… more

Margaret Mitchell
H-index: 50
Computer science 94%
Psychology 5%
mmitchell.bsky.social
Super aligned with this list btw, appreciate your posting.
mmitchell.bsky.social
From the standpoint of being in the tech industry and research community this conference concerns, I'll be looking for what valid scientific results might be drawn from the conference's content, and how it might inform protocols, policies and regulation around the use of these systems.
mmitchell.bsky.social
From ethics standpoint, I can't condone the use of GenAI w/o explicit informed consent. 5/
mmitchell.bsky.social
One thing that both me and Clémentine are noting is that people *are doing this* in actual scientific conferences. Researchers are generating with GenAI, as are reviewers. Policies around this and disclosures are ad-hoc, inconsistent, and uninformed. So, how might this conference relate to that? 4/
mmitchell.bsky.social
I wouldn't call the conference itself "hype" -- it's a concrete thing/experiment, whereas hype refers to the promotion/publicity of it -- so description *of* the conference, or unsupported conclusions *from* the conference, are at risk of overhyping. 3/
mmitchell.bsky.social
Some CONs: Can be misinterpreted, regardless of what happens, as proof that LLM-based systems *can* or *should* be used to generate papers/reviews. It's still GenAI, which takes from from others w/o explicit informed consent. Can result in plagiarism+erasure of the researchers it's regurgitating.2/
mmitchell.bsky.social
Some PROs: It's going to provide a dataset that can be used (by humans) to analyze LLM-based systems for what they do and don't do, and a methodology that can be expanded with different prompts to test behaviors. This can also be used to (eg) help diagnose whether a real reviewer is using an LLM. 1/
mmitchell.bsky.social
Hugging Face (thankfully) doesn't do a groupthink too much -- e.g., "Hugging Face thinks this". We're generally able to have different opinions and thoughts, which is part of the open/collaborative ethos. I don't feel strongly about this conference myself, I see pros and cons. 🧵
mmitchell.bsky.social
Aside: Did the CEO of Tylenol anger the administration? It's odd they're focusing on a specific brand, no? Not the active ingredient, acetaminophen (also called paracetamol)? Is buying Generic Acetaminophen fine, but not Tylenol? This seems to me like there's something else behind the scenes. /end
mmitchell.bsky.social
Here,
- Experimenter bias (and bad science) influences how the severity scale is constructed
- The misunderstanding of the difference between correlation and causation gives rise to a pretty incendiary conclusion
- Which will likely make tons of waves b/c public at large will miss this.
8/
mmitchell.bsky.social
Now let's look at the other example Experimenter Bias Protocol.
- If a "severity" scale is used for this experiment where "severity" incorporates "tendency to self-harm", then people who rank "severe" will have more injuries.
- Increased injuries --> increased Tylenol use.
7/
mmitchell.bsky.social
- Showing a "link" of any sort doesn't show *causation*. Related: Correlation is not causation.
en.wikipedia.org/wiki/Correla...
- If your experiment, by design, links 2 things, then a result showing that they're linked comes from *you linking them*. (That's classic experimenter bias fallacy.)
6/
Correlation does not imply causation - Wikipedia
en.wikipedia.org
mmitchell.bsky.social
The experimental design doesn't disprove the *opposite* idea, that Tylenol use is linked to *no* autism.
I.e., There is no piece of that design that accounts for the fact that the majority of people probably have taken acetaminophen. 5/
mmitchell.bsky.social
In the first protocol, the issues are:
- It starts with ppl w/ autism, not the population at large. The majority of all people have probably taken acetaminophen, so, the majority of people with autism have prob taken acetaminophen. BUT, the majority of ppl *without* autism have prob taken it. 4/
mmitchell.bsky.social
Experimenter bias protocol #2, more 🔥:
1. Take group of people with autism
2. Isolate those who've taken acetaminophen
3. Rank them along a scale of "autism severity"
4. Calculate *how much* each has taken
5. Possible Result: "Increased Severity of Autism Associated with Increased Use of Tylenol"
3/
mmitchell.bsky.social
Experimenter bias protocol #1:
(1) Take a group of people with autism
(2) Calculate how many of them have taken paracetamol/acetaminophen (the active ingredient in Tylenol)
(3) Establish that the majority of them have
(4) Report: "Tylenol linked to almost ALL cases of Autism"
2/
mmitchell.bsky.social
Oh I hope so! It made me laugh out loud, it was such a perfect description of the intention of the play *EXCEPT FOR* the "emotional resonance" bit. That's the bit that made me think it was a legitimate review.
mmitchell.bsky.social
"Experimenter bias" refers to a mistake in scientific studies where the experimenter wants to see something, and so unconsciously designs protocols to prove what they want to see. RFK here provides an example, Let me explain what we might see based on this. 🧵
atrupar.com
RFK Jr on Tylenol and autism: "It is not proof. We're doing the studies to make the proof."
mmitchell.bsky.social
🎭 Come for the Godot, stay for Alex Winter running with so much momentum he makes it to the ceiling.
mmitchell.bsky.social
🎭 As a Samuel Beckett and Alex Winter fan, I highly, highly *highly* recommend going to see this version of Godot. Laugh-out-loud funny, masterfully handled some of the confusions from text. Use of funnel as stage was brilliant. Only props were hats and shoes. Beckett would be proud.
Keanu Reeves and Alex Winter as Gogo and Didi, sitting in a funnel-shaped tan/white stage against a stark black background.
mmitchell.bsky.social
🎭 As a Samuel Beckett and Alex Winter fan, I highly, highly *highly* recommend going to see this version of Godot. Laugh-out-loud funny, masterfully handled some of the confusions from text. Use of funnel as stage was brilliant. Only props were hats and shoes. Beckett would be proud.
Keanu Reeves and Alex Winter as Gogo and Didi, sitting in a funnel-shaped tan/white stage against a stark black background.
mmitchell.bsky.social
Apparently this was Portland last night. Gosh I love my home away from home. ❤️

References

Fields & subjects

Updated 1m