Virtual Room 1: Covariate Balancing

Date: 

Friday, July 17, 2020, 12:00pm to 1:30pm

Balancing covariates in randomized experiments using the Gram-Schmidt Walk

Christopher Harshaw, Fredrik Sävje, Daniel Spielman and Peng Zhang

Kpop: A kernel balancing approach for reducing specification assumptions in survey weighting

Erin Hartman, Chad Hazlett and Ciara Sterbenz

 

Chair: Ludovic Rheault (University of Toronto)

 

Co-Host: Anwar Mohammed (McMaster University)

Balancing covariates in randomized experiments using the Gram-Schmidt Walk

Download Paper

Author(s): Christopher Harshaw, Fredrik Sävje, Daniel Spielman and Peng Zhang

Discussant: Marc Ratkovic (Princeton University)

 

The paper introduces a class of experimental designs that allows experimenters to control the robustness and efficiency of their experiments. The designs build on a recently introduced algorithm in discrepancy theory, the Gram--Schmidt walk. We provide a tight analysis of this algorithm, allowing us to prove important properties of the designs it produces. These designs aim to simultaneously balance all linear functions of the covariates, and the variance of an estimator of the average treatment effect is shown to be bounded by a quantity that is proportional to the loss function of a ridge regression of the potential outcomes on the covariates. No regression is actually conducted, and one may see the procedure as regression adjustment by design. The class of designs is parameterized so to give experimenters control over the worse case performance of the treatment effect estimator. Greater covariate balance is attained by allowing for a less robust design in terms of worst case variance. We argue that the trade-off between robustness and efficiency is an inherent aspect of experimental design. Finally, we provide non-asymptotic tail bounds for the treatment effect estimator under the class of designs we describe.

Kpop: A kernel balancing approach for reducing specification assumptions in survey weighting

Author(s): Erin Hartman, Chad Hazlett and Ciara Sterbenz

Discussant: Luke W. Miratrix (Harvard University)

 

Response rates to surveys have declined precipitously. For example, Pew Research Center saw response rates to telephone surveys fall from roughly one third of respondents in the late 1990s, to only 6% in 2018. Some researchers have responded by relying more heavily on convenience-based internet samples. This leaves researchers asking not if, but how, to weight survey results to represent their target population. Though practitioners often call upon expert knowledge in constructing their auxiliary vector, X, to use in weighting methods, they face difficult, feasibility-constrained choices of what interactions or other functions to include in X. Most approaches seek weights on the sampled units that make measured covariates have the same mean in the sample as in the population. However, the weights that achieve equal means on X will ensure that an outcome variable of interest Y is correctly reweighted only if the expectation of Y is linear in X, an unrealistic assumption. We describe kernel balancing for population reweighting (KPOP) to make samples more similar to populations on the distribution of X, beyond the first moment margin. This approach effectively replaces X with a kernel matrix, K, that encodes high-order information about X via the “kernel trick”. We then reweight the sampled units so that their average row of K is approximately equal to that of the population, working through a spectral decomposition. This produces good calibration on a wide range of smooth functions of X, without relying on the user to select those functions. We describe the method and illustrate its use in reweighting political survey samples, including from the 2016 American presidential election.


Add to Calendar