Priming Bias Versus Post-Treatment Bias in Experimental Designs

Jacob Brown (Harvard University), Matthew Blackwell (Harvard University), Sophie Hill (Harvard University), Kosuke Imai (Harvard University) and Teppei Yamamoto (Massachusetts Institute of Technology)

Abstract:  It is now widely recognized that conditioning on variables affected by a treatment can induce post-treatment bias when estimating causal effects. Although this suggests that researchers should measure moderators prior to the administration of the treatment, doing so may also bias estimated causal effects if the covariate measurement primes respondents by making them react differently to the treatment. In this paper, we formally analyze this tradeoff between pre- and post-treatment measurement of covariates in experiments. We derive bounds for conditional average treatment effects and interactions for covariates measured post-treatment and show how to use substantive assumptions to narrow these bounds. This allows researchers to assess the sensitivity of their empirical findings to a possible priming bias while being free of post-treatment bias. We apply these bounds to an experimental analysis of support for nuclear strikes. We conclude with practical recommendations for scholars designing experiments.

View Poster in a New Tab