Alex Coppock
LightNews LightNews
aecoppock.bsky.social
Alex Coppock
@aecoppock.bsky.social
6.5K followers 1.5K following 550 posts
Associate Professor of Political Science at Northwestern University
alexandercoppock.com
Persuasion in Parallel: https://alexandercoppock.com/coppock_2023.html
Research Design: Declaration, Diagnosis, and Redesign: book.declaredesign.org
Posts Replies Media Videos
aecoppock.bsky.social
(prohibition) persuasion in parallel

Had fun reconstructing the data from this 90-year old persuasion experiment that shows that "wet" and "dry" college students update their attitudes in the direction of counterattitudinal persuasive information.

paper: doi.org/10.1080/0022...
plot of reconstructed data The Speech of man is one of the most obvious of his social activi- tics. O u r educational systems lean heavily on the presumption that speech is a vital tool in the stimulation of individual development. T h e lawyer and theologian carry on their work almost entirely through the use of speech. The zealous protection of the right of freedom of speech in democratic societies may be said to be based on the premise that where men do not enjoy freedom of speech they .soon become slaves. Millions of dollars are spent annually by advertisers with a naive faith in the power of the word to convince men and move them to action. The use of speech by man is a psy- chological phenomenon unique in the world of subjects for scientific rtudy. Yet in spite of all these facts, one finds surprisingly few re- ports in the scientific literature of objective measurements of the actual effects of the use of speech on the social environment. the table that provides the stats
November 19, 2025 at 10:16 PM
aecoppock.bsky.social
I'm one of those academics who measure message persuasiveness via survey experiment. I think it's absolutely the right method for doing so.

I think the response to this good point from @anatosaurus.bsky.social is not to abandon survey exps but instead to measure "getting heard" (attention) also.
At first glance, Pollingism espouses a so-simple-a-child-gets-it truth: ask people what matters to them and repeat it back. It often gets short-handed as “say popular things” and, by extension, eschew unpopular ones. However, this notion is built off of assumptions that do not bear out in the real world. Let’s start with an irrefutable claim: a message that nobody hears cannot persuade them. Nor can it get them to act in opposition to the present regime, vote in an election, or select a specific candidate. For years, we have heard entirely accurate lamentations that Democrats are failing to reach people. To be sure, the right’s far better media system merits every alarm bell, not merely for electoral outcomes but also for our hopes of a functional society, free of lies about everything from vaccines to wind turbines to the dining habits of newcomers. But even setting that imbalance aside, Pollingism’s message creation plus testing model fails to consider, let alone optimize for, how we would get a message to be heard. Presently, the dominant approaches to campaign message crafting and evaluation involve (1) randomized-controlled trial (RCT) testing of messages and/or ads, and (2) issue-based polling to figure out which topics voters rate most vital and therefore merit airtime. But these tools cannot tell us what would break through in the cacophony of political noise to which voters are subjected.
November 5, 2025 at 6:13 PM
aecoppock.bsky.social
lol at the MPSA shade in the last line of the FAQ
Will the deadline be extended a week, like other conferences? No!
November 5, 2025 at 4:52 PM
aecoppock.bsky.social
Maybe your texts tell you to "BE A VOTER" like mine.

It all started because of a PNAS paper that claimed that the noun form it increased voter turnout (relative to the verb form ) by 11 to 14 percentage points.

It keeps not replicating, obviously.

Most recently doi.org/10.1017/bpp....
November 4, 2025 at 6:25 PM
aecoppock.bsky.social
obligatory C&H
the c&H comic
October 31, 2025 at 7:02 PM
aecoppock.bsky.social
Thank you for sharing -- I was unaware of this framework for choosing among replication targets. I think I stand with the critics. Of course I'm left still not knowing how to choose among empirical estimands!

Separately, this figure is amazing:
a good figure that skewers an "anonymous" reviewer for requiring many citations to (presumably) their work
October 30, 2025 at 4:46 PM
aecoppock.bsky.social
chat's personalized flattery is getting out of hand
chat really told me: Excellent — that’s exactly what Hadley would do next.
October 29, 2025 at 3:46 PM
aecoppock.bsky.social
encountered this article by David Weimer from 1986 (cited by 7) with three good ideas for improving science that just a few short decades later, we've started to implement.

Collective Delusion In The Social Sciences: Publishing Incentives For Empirical Abuse

doi.org/10.1111/j.15...
title page of article: There is a skeleton in the social science closet. Almost all of us have heard it rattled at one time or another.! Nevertheless, we collectively ignore it. Some ignore it because they do not think it is very scary: others because its putting to rest would require a major restructuring of our scholarly journals. Perhaps others even fear that admitting its existence would undermine belief in the empirical progress? the social sciences have apparently enjoyed since the widespread diffusion of statistical training and the ready availability of the computer. I will argue that the skeleton, the bias social science journals exhibit for publishing articles reporting statistically significant results, is dangerous The results of statistical tests should not be submitted to or als until after articles have been accepted for publica- tion.. would base their decisions on theory under discussions of the theory under consideration, the specific hypotheses to be tested, and the data sample to be used.? This would divert attention from the final result to the a priori specification of hypotheses and the appropriateness of data and statistical technique for testing them. Unfortunate-ly, this approach would not work unless adopted by all the relevant journals. If a single journal adopted it, researchers Any journal that publishes an article with statistical models should be required (by the disciplines?) to provide a page of space to anyone who wants to report the results of applying the authors' models to different sets of data. This approach would provide a less biased sampling of research results. It would also allow us to better gauge the robustness of our theories.8 Some journals now provide some space for veri-fications; doing so should be standard editorial practice. Researchers should be required to submit a statement with indicating their articles indicating whether or not the model being presented is the one actually first estimated from the data. A general description of statistical techniques would also be included in the submission. Is the presented model the first specification or is it the end product of stepwise regression or some other questionable data search? Where appropriate, the editor would affix a warning along the following lines next to results:
October 27, 2025 at 12:28 PM
aecoppock.bsky.social
👀 this new meta-analysis on edutainment by @bardiarahmani.bsky.social, Montano, @dylanwgroves.bsky.social, and Green

doi.org/10.1017/bpp....

377 ests in 77 exps: edutainment moves attitudes, norms, beliefs 📊
Effects persist ⏳
Many reasonable theories about effect heterogeneity are not supported 😇
October 14, 2025 at 5:40 PM
aecoppock.bsky.social
this part seems cool
Guided by our values and theory of policy design, Searchlight’s approach to policy will be defined by bold, persistent experimentation.
September 17, 2025 at 2:06 PM
aecoppock.bsky.social
love this -- so very useful.

btw you *can* write your own handler as you did here (which is v. flexible and extensible) but you can also do many post-estimation things with the .summary argument if you like!
September 14, 2025 at 2:56 PM
aecoppock.bsky.social
whaaaat!? that's so cool!

literally just:

weird_trick = D * X

iv_robust(weird_trick ~ D | Z)

to get the mean of X among compliers!
design declaration for one weird trick design diagnosis
September 3, 2025 at 7:27 PM
aecoppock.bsky.social
This post seriously conflates "polling" with "survey experimentation"

Also any paragraph that starts this way is a big 🚩
I know that correlation does not equal causation. But
August 27, 2025 at 3:14 PM
aecoppock.bsky.social
I
That’s a beautiful and uncanny concept—very Borges. Here's a short Borges-style story based on your idea, titled “The Final Word.” The Final Word by a humble imitator of Borges It began, as most obsessions do, with an aspiration toward clarity. Dr. Ezequiel Varas, a semi-retired philologist of uncertain reputation, had long maintained that the proliferation of knowledge had not enlightened the world but drowned it in multiplicity. He lamented, in letters never sent, that to know everything was to know nothing in particular. In the autumn of his seventy-fourth year, Dr. Varas gained access to a powerful artificial intelligence—an inheritance from a distant cousin in Palo Alto who had drowned in the Pacific under ambiguous circumstances. The machine—named Pleroma, from the Greek for fullness—had been trained on every known language, book, map, film, genetic sequence, war diary, weather report, and dream journal ever digitized. Dr. Varas asked Pleroma for a summary of the world. The machine obliged. It took the form of a vast document, thousands of pages long, which he printed on thin paper and read over the course of a year. The text was perfect. Nothing essential, it seemed, had been omitted. Still, the burden of its volume gnawed at him. He asked Pleroma to summarize the summary. Then again. Then again. With each recursion, the world grew more compressed, more potent. The thirteenth summary fit on a single page. The sixteenth, a single paragraph. The nineteenth, a single sentence. The twenty-second, a single word. The word was: ALETHEIA Ancient Greek: “truth” or “unconcealment.” He asked Pleroma what it meant, but the machine refused. “It is the whole,” it said. “To define it is to break it.” Varas, unbothered, wrote the word on a small slip of paper, burned the rest of the documents, and locked himself in his study. Neighbors sometimes saw the dim light at night and heard him repeating the word aloud. He did not eat, nor did he sleep, but he appeared…
July 14, 2025 at 3:48 PM
aecoppock.bsky.social
With @mollyow.bsky.social, we wrote up an email exchange about testing for causal effect heterogeneity with the grf package. In short, a recommended approach had v. bad properties (wild over-rejection of true nulls of homogeneity) but Molly had a fix.

alexandercoppock.com/testing_with...
using the bad approach using Molly's good approach
June 26, 2025 at 8:18 PM
aecoppock.bsky.social
This new package for mediation is great! The user interface is *so intuitive* but the unsung hero of this software is the documentation:
Arguments data Not needed, wouldn't make your analysis and more correct. Details We have: X: a binary treatment variable, that is affected by an unmeasured confounder (U). M: a mediator, that is mostly driven by U, not X Y: an outcome, that is influenced by both M and U, but you can’t see U. What you don’t have: Any valid identification strategy Temporal ordering Careful causal reasoning Value Nothing, because you shouldn't do mediation analysis.
May 6, 2025 at 2:53 PM
aecoppock.bsky.social
whoa! Some mild dependence of the SE of the "both" estimator over the full range of rho...
full range of rho
April 29, 2025 at 7:48 PM
aecoppock.bsky.social
had some fun simulating this one in DeclareDesign. outcome model is Y ~ 0.5 * D + 0.5*X_1 + 0.5*X_2 + U with D confounded by X_1 and X_2. gotta control for both or BIAS. Simulation agrees with you, no precision loss (on ATE estimate) in this setup when X_1 and X_2 are v. correlated.
DD code DD simulation figure
April 29, 2025 at 5:55 PM
aecoppock.bsky.social
I appreciate your perspective here Dan and agree that data viz can easily obscure inferences

With my visualizations,

*what* I'm trying to convey is the experimental design and the results

*to whom* I'm trying to communicate is the community of scientists
abstract for VAYR
April 24, 2025 at 12:27 PM
aecoppock.bsky.social
Congratulations @awilke.bsky.social!

Anna appears to a tables > figures kind of scholar so no screenshots of results -- you'll have to click through to actually read this paper to learn what happened :)

doi.org/10.1111/ajps...
I implemented a field experiment inSouth Africa that randomly varies the capacity of police to locate households.Findings from surveys conducted several months later suggest households that have become legible to police are more willing to rely on police and less willing to participate in vigilantism
April 22, 2025 at 3:25 PM
aecoppock.bsky.social
We don't need new software for most of these recommendations, we just need to use existing tools in ways that respect the design.

A major exception is how to deal with overplotting, since experiments often feature discrete treatments and discrete outcomes (binary or likert scales), e.g.:
figure from persuasion in parallel
April 16, 2025 at 7:53 PM
aecoppock.bsky.social
Like all good package names, this one needs some explanation:

vayr stands for Visualize As You Randomize, which is a philosophy for design-based graphs.

The basic idea is that the visualization should both convey and respect the experimental design.

alexandercoppock.com/coppock_2020...
A good statistical graph for a randomized experiment simultaneously conveys the study's design, analysis, and results. It reveals the experimental design by mapping design elements to aesthetic parameters. It illuminates the analysis by plotting the statistical model in “dataspace.” When the design and analysis of an experiment are encoded in a plot, the interpretation of the experimental results is clarified. “Analyze as you randomize” is a dictum attributed to Fisher that guides interpretations of experimental data. This chapter extends that principle to visualizations of randomized experiments. While not every experiment requires a visualization, those that do should be visualized in ways that communicate the design and results together.
April 16, 2025 at 7:53 PM
aecoppock.bsky.social
New visualization tool alert!

The vayr package version 1.0.0 is now on CRAN.

It contains position adjustments for ggplot2 that help with overplotting in pleasing ways. My favorite is position_sunflower().

- install.packages("vayr")
- alexandercoppock.com/vayr

#rstats #ggplot2 #dataviz
position_jitter() position_jitter_ellipse() position_sunflower() position_circlepack()
April 16, 2025 at 7:53 PM
aecoppock.bsky.social
TIL, the original meaning of "conjoint" was about the "conjoining" of multiple attributes, not multiple profiles!

www.jstor.org/stable/31495...

That said, I agree there's no serious design distinction between single profile conjoint experiments and factorial experiments.
April 8, 2025 at 6:27 PM
aecoppock.bsky.social
One picture I've made shows the 21 possible random assignments in Chapter 3 of Gerber and Green like the left figure (treat 2 of 7). the right figure shows the sampling distributions by other assignment schemes. kinda neat?
all 21 possible assignments six sampling distributions
April 7, 2025 at 4:20 PM