Virtual Room 2: Conjoint Designs

Date: 

Friday, July 17, 2020, 12:00pm to 2:15pm

Avoiding Measurement Error Bias in Conjoint Analysis

Katherine Clayton, Yusaku Horiuchi, Aaron Kaufman, Gary King and Mayya Komisarchik

Double Feature
Improving Preference Elicitation in Conjoint Designs using Machine Learning for Heterogeneous Effects

Scott F. Abramson, Korhan Kocak, Asya Magazinnik and Anton Strezhnev

Using Conjoint Experiments to Analyze Elections: The Essential Role of the Average Marginal Component Effect (AMCE)

Kirk Bansak, Jens Hainmueller, Daniel Hopkins and Teppei Yamamoto

 

Chair: Peter J. Loewen (University of Toronto)

 

Co-Host: Md Mujahedul Islam (University of Toronto)

Avoiding Measurement Error Bias in Conjoint Analysis

Author(s): Katherine Clayton, Yusaku Horiuchi, Aaron Kaufman, Gary King and Mayya Komisarchik

Discussant: Naoki Egami (Columbia University)

 

Conjoint analysis is a survey research methodology spreading fast across the social sciences and marketing due to its widespread applicability and apparent capacity to disentangle many causal effects with a single survey experiment. Unfortunately, conjoint designs are also especially prone to measurement error, revealed by surprisingly low levels of intra-coder reliability, which can exaggerate, attenuate, or give incorrect signs for causal effect estimates. We show that measurement error bias is endemic in applications, and so assuming its absence, as many studies implicitly do, is not defensible. With replications of prior research and new experiments, we demonstrate three common mechanisms that generate measurement error. We use these mechanisms to design open source software to help researchers design conjoint experiments, study the effects of measurement error, and correct for the resulting biases.

DOUBLE FEATURE

Improving Preference Elicitation in Conjoint Designs using Machine Learning for Heterogeneous Effects

Download Paper

Author: Scott F. Abramson, Korhan Kocak, Asya Magazinnik and Anton Strezhnev

Discussant: Jasjeet Sekhon (University of California, Berkeley)

 

Conjoint analysis has become a standard tool for preference elicitation in political science. However the typical estimand, the Average Marginal Component Effect (AMCE), is only tangentially linked to theoretically relevant quantities. In this paper we clarify the necessary theoretical assumptions to interpret the AMCE in terms of individual preferences, explain how heterogeneity in marginal component effects can drive misleading conclusions about preferences, and provide a set of tools based on the causal/generalized random forest method (Athey et al., 2019; Wager & Athey, 2018) that allow applied researchers to detect effect heterogeneity between respondents and derive theoretically relevant quantities of interest from estimates of individual-level marginal component effects. We illustrate this method with an application to a recently conducted conjoint experiment on candidate preferences in the 2020 U.S. Democratic Presidential primary.

Using Conjoint Experiments to Analyze Elections: The Essential Role of the Average Marginal Component Effect (AMCE)

Download Paper

Author(s): Kirk Bansak, Jens Hainmueller, Daniel Hopkins and Teppei Yamamoto

Discussant: Kosuke Imai (Harvard University)

 

Political scientists have increasingly deployed conjoint survey experiments to understand multi-dimensional choices in various settings. We begin with a general framework for analyzing voter preferences in multi-attribute elections using conjoints. With this framework, we demonstrate that the Average Marginal Component Effect (AMCE) is well-defined in terms of individual preferences and represents a central quantity of interest to empirical scholars of elections: the effect of a change in an attribute on a candidate or party's expected vote share. This property holds irrespective of the heterogeneity, strength, or interactivity of voters' preferences and regardless of how votes are aggregated into seats. Overall, our results indicate the essential role of AMCEs for understanding elections, a conclusion buttressed by a corresponding literature review. We also provide practical advice on interpreting AMCEs and discuss how conjoint data can be used to estimate other quantities of interest to electoral studies.


Add to Calendar