Roger Levy
@rplevy.bsky.social
Director, MIT Computational Psycholinguistics Lab. President, Cognitive Science Society. Chair of the MIT Faculty. Open access & open science advocate. He.
Lab webpage: http://cpl.mit.edu/
Personal webpage: https://www.mit.edu/~rplevy
Lab webpage: http://cpl.mit.edu/
Personal webpage: https://www.mit.edu/~rplevy
Reposted by Roger Levy
New work to appear @ TACL!
Language models (LMs) are remarkably good at generating novel well-formed sentences, leading to claims that they have mastered grammar.
Yet they often assign higher probability to ungrammatical strings than to grammatical strings.
How can both things be true? 🧵👇
Language models (LMs) are remarkably good at generating novel well-formed sentences, leading to claims that they have mastered grammar.
Yet they often assign higher probability to ungrammatical strings than to grammatical strings.
How can both things be true? 🧵👇
November 10, 2025 at 10:11 PM
New work to appear @ TACL!
Language models (LMs) are remarkably good at generating novel well-formed sentences, leading to claims that they have mastered grammar.
Yet they often assign higher probability to ungrammatical strings than to grammatical strings.
How can both things be true? 🧵👇
Language models (LMs) are remarkably good at generating novel well-formed sentences, leading to claims that they have mastered grammar.
Yet they often assign higher probability to ungrammatical strings than to grammatical strings.
How can both things be true? 🧵👇
Reposted by Roger Levy
thinking of calling this "The Illusion Illusion"
(more examples below)
(more examples below)
December 1, 2024 at 2:33 PM
thinking of calling this "The Illusion Illusion"
(more examples below)
(more examples below)
Been listening a lot to Ella Jenkins the last couple of days. What wonderful music and performance!
November 11, 2024 at 11:44 PM
Been listening a lot to Ella Jenkins the last couple of days. What wonderful music and performance!
Within the past two weeks I deleted apps for media companies owned by 67% of the world’s richest billionaires, and it felt great!
November 10, 2024 at 12:47 PM
Within the past two weeks I deleted apps for media companies owned by 67% of the world’s richest billionaires, and it felt great!
Results of high stakes elections that happen only once every four years offer remarkable opportunities for overfitting theories of the electorate
November 8, 2024 at 7:46 PM
Results of high stakes elections that happen only once every four years offer remarkable opportunities for overfitting theories of the electorate
Reposted by Roger Levy
It's my book's release day! The Patterns of Comics is now officially published, featuring an extended data-driven analysis of the structures used in 350+ comics from Asia, Europe, and North America analyzing diversity, regularity, and change over time www.visuallanguagelab.com/poc
December 28, 2023 at 1:18 PM
It's my book's release day! The Patterns of Comics is now officially published, featuring an extended data-driven analysis of the structures used in 350+ comics from Asia, Europe, and North America analyzing diversity, regularity, and change over time www.visuallanguagelab.com/poc
Reposted by Roger Levy
GOOD MORNING BLUESKY!
Very excited about this new paper:
www.pnas.org/doi/10.1073/pnas.2300671120
Key Q: what predicts how much young kids (👶)talk?
How much 🗣 kids heard predicted how much 👶talked, but other factors, e.g. mom’s education, didn’t. #PsychSci #DevPsy 🗣💬
INCOMING SUMMARY🧵ALERT 1/14
Very excited about this new paper:
www.pnas.org/doi/10.1073/pnas.2300671120
Key Q: what predicts how much young kids (👶)talk?
How much 🗣 kids heard predicted how much 👶talked, but other factors, e.g. mom’s education, didn’t. #PsychSci #DevPsy 🗣💬
INCOMING SUMMARY🧵ALERT 1/14
December 13, 2023 at 2:52 PM
GOOD MORNING BLUESKY!
Very excited about this new paper:
www.pnas.org/doi/10.1073/pnas.2300671120
Key Q: what predicts how much young kids (👶)talk?
How much 🗣 kids heard predicted how much 👶talked, but other factors, e.g. mom’s education, didn’t. #PsychSci #DevPsy 🗣💬
INCOMING SUMMARY🧵ALERT 1/14
Very excited about this new paper:
www.pnas.org/doi/10.1073/pnas.2300671120
Key Q: what predicts how much young kids (👶)talk?
How much 🗣 kids heard predicted how much 👶talked, but other factors, e.g. mom’s education, didn’t. #PsychSci #DevPsy 🗣💬
INCOMING SUMMARY🧵ALERT 1/14
#linguistics Bluesky: what are the best available quantitative measures of dialect/language mutual intelligibility? The more fine-grained, the better: I'm hoping to vividly illustrate at least one specific dialect continuum (e.g., the Romance languages of the Mediterranean coast)
December 10, 2023 at 4:17 PM
#linguistics Bluesky: what are the best available quantitative measures of dialect/language mutual intelligibility? The more fine-grained, the better: I'm hoping to vividly illustrate at least one specific dialect continuum (e.g., the Romance languages of the Mediterranean coast)
Reposted by Roger Levy
Today in linguists are NOT KIDDING when we say that your capacity for language enables you to understand sentences that have never before been uttered in human history.
November 20, 2023 at 8:54 PM
Today in linguists are NOT KIDDING when we say that your capacity for language enables you to understand sentences that have never before been uttered in human history.
Reposted by Roger Levy
New postdoc opportunity to work jointly with @cantlonlab.bsky.social and me to understand cognition across species, age, and culture! cmu.wd5.myworkdayjobs.com/CMU/job/Pitt...
November 15, 2023 at 5:26 PM
New postdoc opportunity to work jointly with @cantlonlab.bsky.social and me to understand cognition across species, age, and culture! cmu.wd5.myworkdayjobs.com/CMU/job/Pitt...
Absolutely, the Nature EiC has it completely backwards. Checking for errors and quality of data (and of math, code, and argumentation) is the most important work that reviewers can do.
The quotes from Nature EiC Magdalena Skipper about whether journals should be checking for errors/data quality as part of peer review are quite surprising to me.
https://www.wsj.com/science/whats-wrong-with-peer-review-e5d2d428?st=dhrnljoa74fujcv&reflink=desktopwebshare_permalink
https://www.wsj.com/science/whats-wrong-with-peer-review-e5d2d428?st=dhrnljoa74fujcv&reflink=desktopwebshare_permalink
November 11, 2023 at 12:18 AM
Absolutely, the Nature EiC has it completely backwards. Checking for errors and quality of data (and of math, code, and argumentation) is the most important work that reviewers can do.
Reposted by Roger Levy
The quotes from Nature EiC Magdalena Skipper about whether journals should be checking for errors/data quality as part of peer review are quite surprising to me.
https://www.wsj.com/science/whats-wrong-with-peer-review-e5d2d428?st=dhrnljoa74fujcv&reflink=desktopwebshare_permalink
https://www.wsj.com/science/whats-wrong-with-peer-review-e5d2d428?st=dhrnljoa74fujcv&reflink=desktopwebshare_permalink
November 11, 2023 at 12:11 AM
The quotes from Nature EiC Magdalena Skipper about whether journals should be checking for errors/data quality as part of peer review are quite surprising to me.
https://www.wsj.com/science/whats-wrong-with-peer-review-e5d2d428?st=dhrnljoa74fujcv&reflink=desktopwebshare_permalink
https://www.wsj.com/science/whats-wrong-with-peer-review-e5d2d428?st=dhrnljoa74fujcv&reflink=desktopwebshare_permalink
Reposted by Roger Levy
I am delighted to announce that the Department of Biology at the University of Washington is advertising for a tenure-track assistant professor position on the quantitative understanding of collective behavior.
I will be chairing the search; details are here: apply.interfolio.com/130336
I will be chairing the search; details are here: apply.interfolio.com/130336
November 7, 2023 at 11:20 PM
I am delighted to announce that the Department of Biology at the University of Washington is advertising for a tenure-track assistant professor position on the quantitative understanding of collective behavior.
I will be chairing the search; details are here: apply.interfolio.com/130336
I will be chairing the search; details are here: apply.interfolio.com/130336
Reposted by Roger Levy
I think it’s amazing that Cognitive Science gives recent PhDs $10K in UNRESTRICTED CASH…right when folks are broke, exhausted, moving town…and need it most.
It’s almost Glushko season!
cognitivesciencesociety.org/glushko-diss...
It’s almost Glushko season!
cognitivesciencesociety.org/glushko-diss...
Glushko Dissertation Prize - Cognitive Science Society
cognitivesciencesociety.org
November 7, 2023 at 12:41 PM
I think it’s amazing that Cognitive Science gives recent PhDs $10K in UNRESTRICTED CASH…right when folks are broke, exhausted, moving town…and need it most.
It’s almost Glushko season!
cognitivesciencesociety.org/glushko-diss...
It’s almost Glushko season!
cognitivesciencesociety.org/glushko-diss...
Reposted by Roger Levy
First post and big news - I am starting as an Assistant Professor in Psychology at Georgia Tech in Jan 2024!
www.language-intelligence-thought.net
www.language-intelligence-thought.net
LIT Lab
Our goal is to understand the relationship between language and human thought.
How does the language network in the brain interact with other systems to interpret meaning in the world?
Can models...
www.language-intelligence-thought.net
November 6, 2023 at 4:52 PM
First post and big news - I am starting as an Assistant Professor in Psychology at Georgia Tech in Jan 2024!
www.language-intelligence-thought.net
www.language-intelligence-thought.net
Reposted by Roger Levy
In a new TiCS article, @emaliemcmahon.bsky.social and I review a growing body of behavioral, neural, and computational evidence that social interactions are automatically extracted by the human visual system:
tinyurl.com/nhh2dhxt
#PsychSciSky #NeuroSkyence
tinyurl.com/nhh2dhxt
#PsychSciSky #NeuroSkyence
In our new paper out today at Trends in Cognitive Science, @lisik.bsky.social and I argue that social interaction perception is a visual process–computed by the visual system. (1/3)
tinyurl.com/nhh2dhx
#PsychSciSky #CogSci #CogPsyc #compneuro
tinyurl.com/nhh2dhx
#PsychSciSky #CogSci #CogPsyc #compneuro
October 5, 2023 at 2:22 PM
In a new TiCS article, @emaliemcmahon.bsky.social and I review a growing body of behavioral, neural, and computational evidence that social interactions are automatically extracted by the human visual system:
tinyurl.com/nhh2dhxt
#PsychSciSky #NeuroSkyence
tinyurl.com/nhh2dhxt
#PsychSciSky #NeuroSkyence
Reposted by Roger Levy
While the world has its eyes on the Middle East, democratic conditions in Indonesia are looking grim. The Supreme Court has overruled the Constitution in order to allow the sitting president's son to stand as Vice Presidential candidate with a disgraced general with a stained human rights record
November 3, 2023 at 11:06 PM
While the world has its eyes on the Middle East, democratic conditions in Indonesia are looking grim. The Supreme Court has overruled the Constitution in order to allow the sitting president's son to stand as Vice Presidential candidate with a disgraced general with a stained human rights record
Reposted by Roger Levy
I am reading PhD applications this year, with a special interest in students who would like to work on the topic of perceived danger. But open to all applicants who share some of my interests. Visit www.liulaboratory.org to see papers, lab values, and tips for application writing.
LIU LAB
In the Look, Infer, and Understand (LIU) Lab at Johns Hopkins University, we are interested in how our minds and brains reason about the physical and social world. We study the developmental and neu...
www.liulaboratory.org
November 2, 2023 at 5:27 PM
I am reading PhD applications this year, with a special interest in students who would like to work on the topic of perceived danger. But open to all applicants who share some of my interests. Visit www.liulaboratory.org to see papers, lab values, and tips for application writing.
Reposted by Roger Levy
The Consumer Financial Protection Bureau (CFPB) is hiring a section chief for the psych/”behavioral” section and I'd love to see some CogSci representation in there! My brother works there (trained as an economist) and it's an incredible gig doing research in the public interest.
Supervisory Research Scientist (Interdisciplinary)
This position is located at the Consumer Financial Protection Bureau (CFPB), Office of Research. The incumbent supervises and conducts independent, self-directed social/behavioral analysis on a variet...
www.usajobs.gov
November 3, 2023 at 6:22 PM
The Consumer Financial Protection Bureau (CFPB) is hiring a section chief for the psych/”behavioral” section and I'd love to see some CogSci representation in there! My brother works there (trained as an economist) and it's an incredible gig doing research in the public interest.
Reposted by Roger Levy
Santa Fe Institute now has a Blue Sky account: @sfiscience.bsky.social
November 3, 2023 at 5:43 PM
Santa Fe Institute now has a Blue Sky account: @sfiscience.bsky.social
Reposted by Roger Levy
A new cross-linguistic study on demonstratives by a team of psychologists and linguists: "Commonalities and differences across languages in spatial communication can be understood in terms of universal constraints on action shaping spatial language and cognition." www.nature.com/articles/s41...
Spatial communication systems across languages reflect universal action constraints - Nature Human B...
Coventry et al. show that spatial demonstratives—such as ‘this’ and ‘that’ in English—are selected on the basis of whether the speaker is able to reach the object or not, across 29 diverse...
www.nature.com
October 31, 2023 at 7:12 PM
A new cross-linguistic study on demonstratives by a team of psychologists and linguists: "Commonalities and differences across languages in spatial communication can be understood in terms of universal constraints on action shaping spatial language and cognition." www.nature.com/articles/s41...
Reposted by Roger Levy
New paper out!
"Large language models show human-like content biases in transmission chain experiments"
#CulturalEvolution #cssky 🧪
www.pnas.org/doi/10.1073/...
"Large language models show human-like content biases in transmission chain experiments"
#CulturalEvolution #cssky 🧪
www.pnas.org/doi/10.1073/...
PNAS
Proceedings of the National Academy of Sciences (PNAS), a peer reviewed journal of the National Academy of Sciences (NAS) - an authoritative source of high-impact, original research that broadly spans...
www.pnas.org
October 27, 2023 at 7:27 AM
New paper out!
"Large language models show human-like content biases in transmission chain experiments"
#CulturalEvolution #cssky 🧪
www.pnas.org/doi/10.1073/...
"Large language models show human-like content biases in transmission chain experiments"
#CulturalEvolution #cssky 🧪
www.pnas.org/doi/10.1073/...
Reposted by Roger Levy
new work just dropped, see @stephan-meylan.bsky.social's "thread" below:)
#DevPsych #CogPsych #PsychSciSky #CogSciSky
#DevPsych #CogPsych #PsychSciSky #CogSciSky
How do adults understand children’s early, highly variable speech? Our new paper in Nature Human Behavior (www.nature.com/articles/s41...) provides evidence that adults’ interpretations depend quite strongly on language expectations—what they think children are likely to say. 1/
October 26, 2023 at 8:55 PM
new work just dropped, see @stephan-meylan.bsky.social's "thread" below:)
#DevPsych #CogPsych #PsychSciSky #CogSciSky
#DevPsych #CogPsych #PsychSciSky #CogSciSky
Thrilled at publication of
@stephan-meylan.bsky.social's "How adults understand what young children say", featuring Bayesian noisy-channel inference, LLMs, & child speech datasets!
TL;DR: prior expectations of what kids *want to say* is crucial. (Knowing how kids mispronounce words is too.)
@stephan-meylan.bsky.social's "How adults understand what young children say", featuring Bayesian noisy-channel inference, LLMs, & child speech datasets!
TL;DR: prior expectations of what kids *want to say* is crucial. (Knowing how kids mispronounce words is too.)
October 26, 2023 at 9:00 PM
Thrilled at publication of
@stephan-meylan.bsky.social's "How adults understand what young children say", featuring Bayesian noisy-channel inference, LLMs, & child speech datasets!
TL;DR: prior expectations of what kids *want to say* is crucial. (Knowing how kids mispronounce words is too.)
@stephan-meylan.bsky.social's "How adults understand what young children say", featuring Bayesian noisy-channel inference, LLMs, & child speech datasets!
TL;DR: prior expectations of what kids *want to say* is crucial. (Knowing how kids mispronounce words is too.)
Prompting is *not a substitute* for probability measurements in large language models – the amazing
@jennhu.bsky.social's EMNLP paper now available camera-ready!
TL;DR: direct probability measurements show LLMs' linguistic generalizations are better than suggested by prompt-based tests.
@jennhu.bsky.social's EMNLP paper now available camera-ready!
TL;DR: direct probability measurements show LLMs' linguistic generalizations are better than suggested by prompt-based tests.
To researchers doing LLM evaluation: prompting is *not a substitute* for direct probability measurements. Check out the camera-ready version of our work, to appear at EMNLP 2023! (w/ @rplevy.bsky.social)
Paper: arxiv.org/abs/2305.13264
Original thread: twitter.com/_jennhu/stat...
Paper: arxiv.org/abs/2305.13264
Original thread: twitter.com/_jennhu/stat...
October 24, 2023 at 4:40 PM
Prompting is *not a substitute* for probability measurements in large language models – the amazing
@jennhu.bsky.social's EMNLP paper now available camera-ready!
TL;DR: direct probability measurements show LLMs' linguistic generalizations are better than suggested by prompt-based tests.
@jennhu.bsky.social's EMNLP paper now available camera-ready!
TL;DR: direct probability measurements show LLMs' linguistic generalizations are better than suggested by prompt-based tests.