Virtual Room 2: Causal Inference


Wednesday, July 15, 2020, 12:00pm to 1:30pm


Chair: Kenichi Ariga (University of Toronto)


Co-Host: Ilayda Onder (Penn State University)

Casual Inference, or How I Learned to Stop Worrying and Love Hypothesis Testing

Author(s): Justin Esarey, Scott de Marchi and Joseph K. Young

Discussant: Luke Keele (University of Pennsylvania)


Many social scientists now consider it necessary for an empirical research design to achieve identification of a causal relationship as defined by the Rubin (1974) causal model or the closely related Pearl (2009) model. In this paper, we argue that no empirical estimand can take a meaningful causal interpretation without a supporting theoretical structure, even if that estimand is strongly identified by a careful research design; that is, an identified research design is necessary but not sufficient for a causal inference. An atheoretical estimand might be ``causal'' in the narrow sense that changes in the dependent variable are ascribable to the treatment in the specific data used in the study, but not in the sense of providing predictive or explanatory guidance for treatment effects in any other situation in the past, present, or future. For instance, when objects of study strategically interact with one another the straightforward application of common causal inference research designs will yield misleading results. To summarize our argument, there can be NO CAUSATION WITHOUT EXPLANATION.

Variation in impacts of letters of recommendation on college admissions decisions: Approximate balancing weights for treatment effect heterogeneity in observational studies

Download Paper

Author(s): Eli Ben-Michael, Avi Feller and Jesse Rothstein

Discussant: Chad Hazlett (UCLA)


Assessing treatment effect variation in observational studies is challenging because differences in estimated impacts across subgroups reflect both differences in impacts and differences in covariate balance. Our motivating application is a UC Berkeley pilot program for letters of recommendation in undergraduate admissions: we are interested in estimating the differential impacts for under-represented applicants and applicants with differing a priori probability of admission. We develop balancing weights that directly optimize for “local balance” within subgroups while maintaining global covariate balance between treated and control populations. We then show that this approach has a dual representation as a form of inverse propensity score weighting with a hierarchical propensity score model. In the UC Berkeley pilot study, our proposed approach yields excellent local and global balance, unlike more traditional weighting methods, which fail to balance covariates within subgroups. We find that the impact of letters of recommendation increases with the predicted probability of admission, with mixed evidence of differences for under-represented minority applicants.

Add to Calendar