Megastudies Improve the Impact of Applied Behavioural Science

Megastudies Improve the Impact of Applied Behavioural Science

Published: Dec 08, 2021
Publisher: Nature (online ahead of print)
Download
Authors

Katherine L. Milkman

Dena Gromet

Hung Ho

Joseph S. Kay

Timothy W. Lee

Pepi Pandiloski

Yeji Park

Aneesh Rai

Max Bazerman

John Beshears

Lauri Bonacorsi

Colin Camerer

Edward Chang

Gretchen Chapman

Robert Cialdini

Hengchen Dai

Lauren Eskreis-Winkler

Ayelet Fishbach

James J. Gross

Samantha Horn

Alexa Hubbard

Steven J. Jones

Dean Karlan

Erika Kirgios

Joowon Klusowski

Ariella Kristal

Rahul Ladhania

George Loewenstein

Jens Ludwig

Barbara Mellers

Sendhil Mullainathan

Silvia Saccardo

Jann Spiess

Gaurav Suri

Joachim H. Talloen

Jamie Taxer

Yaacov Trope

Lyle Ungar

Kevin G. Volpp

Ashley Whillans

Jonathan Zinman

Angela L. Duckworth

Policy-makers are increasingly turning to behavioural science for insights about how to improve citizens’ decisions and outcomes. Typically, different scientists test different intervention ideas in different samples using different outcomes over different time intervals. The lack of comparability of such individual investigations limits their potential to inform policy. Here, to address this limitation and accelerate the pace of discovery, we introduce the megastudy—a massive field experiment in which the effects of many different interventions are compared in the same population on the same objectively measured outcome for the same duration. In a megastudy targeting physical exercise among 61,293 members of an American fitness chain, 30 scientists from 15 different US universities worked in small independent teams to design a total of 54 different four-week digital programmes (or interventions) encouraging exercise. We show that 45% of these interventions significantly increased weekly gym visits by 9% to 27%; the top-performing intervention offered microrewards for returning to the gym after a missed workout. Only 8% of interventions induced behaviour change that was significant and measurable after the four-week intervention. Conditioning on the 45% of interventions that increased exercise during the intervention, we detected carry-over effects that were proportionally similar to those measured in previous research. Forecasts by impartial judges failed to predict which interventions would be most effective, underscoring the value of testing many ideas at once and, therefore, the potential for megastudies to improve the evidentiary value of behavioural science.

How do you apply evidence?

Take our quick four-question survey to help us curate evidence and insights that serve you.

Take our survey