Can Survey Design Reduce Anchoring Bias In Recall Data

Can survey design reduce anchoring bias in recall data? Evidence from Malawi

Recall biases in retrospective survey data are widely considered to be pervasive and have important implications for effective agricultural research. In this paper, we leverage the survey design literature and test three strategies to attenuate mental anchoring in retrospective data collection: question order effects, retrieval cues, and aggregate (community) anchoring. We embed a survey design experiment in a longitudinal survey of smallholder farmers in Malawi and focus on anchoring bias in maize production and happiness exploiting differences between recalled and concurrent responses. We find that asking for retrospective data before concurrent data reduces recall bias by approximately 34% for maize production, a meaningful improvement with no increase in survey data collection costs. Retrieval cues are less successful in reducing the bias for maize reports and involve more data collection time, while community anchors can exacerbate the bias. Reversing the order of questions and retrieval cues do not help to ease the bias for happiness reports.

Can Survey Design Reduce Anchoring Bias in Recall Data?

Anchoring Bias in Recall Data

Understanding the magnitude and source of measurement biases in self-reported data is critical to effective economic policy research. This paper examines the role of anchoring bias in self-reports of objective and subjective outcomes under recall. The research exploits a unique panel survey data set collected over a three-year period from four countries in Central America. It assesses whether respondents use their reported value of specific measures from the most recent survey period as a cognitive heuristic when recalling the value from a previous period, while controlling for the value they reported earlier. We find strong evidence of sizable anchoring bias in self-reported retrospective indicators for both objective measures (household and per capita income, wages, and hours spent on the household’s main activity) and subjective measures (reports of happiness, health, stress, and well-being). In general, we also observe a larger bias in response to negative changes for objective indicators and a larger bias in response to positive changes for subjective indicators.