Sensitivity Analysis for Outcome Tests

Elisha Cohen (Emory University)

Abstract: Outcome tests, a method that can be used for evaluating bias in selection making processes, are especially useful when using administrative datasets that contain only observations after the selection process has occurred. I show the outcome test lower bound derived by Knox, Lowe, and Mummolo (2019), can be adapted for use with risk ratios and hence extended to include an E-value style (Ding and VanderWeele, 2016) sensitivity analysis adjustment. Using data on all settlements paid by the Chicago Police Department from 1985-2015 I estimate a robust lower bound on gender bias. Using my sensitivity adjusted outcome test, I estimate at least 7.9 % of men would not have been hired had they been women.

View Poster in a New Tab