Since 2016, the American Causal Inference Conference (ACIC) has hosted a data challenge in which teams compete to estimate causal impacts in simulated data sets based on real-world data from fields such as health care or education. The competition is a proving ground for cutting-edge causal inference methods that have the potential to transform program evaluation.
Mathematica’s commitment to building and nurturing evidence communities, using the best possible data, methods, and evidence to improve lives, makes a partnership with ACIC a natural fit. Mathematica is proud to organize this year’s competition, which launches on February 15 when the simulated data sets are posted on the data challenge website. Submissions are due April 15, and results will be announced at ACIC 2022 on May 24-25.
“We are thrilled to organize this event, thanks to funding from the Centers for Medicare & Medicaid Services,” said Mariel Finucane, principal statistician. “Causal inference is all about disentangling simple correlation from true causation, which is core to figuring out which social policies actually move the needle for the individuals and communities they serve. We hope that this year’s competition will help us understand which methods give the most precise and nuanced estimates of policy impacts, ultimately ensuring that we’re using the best tools to further our mission of improving public well-being.”
The Mathematica team designed the data sets for the 2022 challenge to mirror data from evaluations of large-scale U.S. health care system interventions that seek to lower Medicare expenditures. They share features of these evaluations that challenge existing causal inference methods.
- Participation in the intervention is not randomly assigned, making it hard to tell whether the patients or practices that were motivated to join the intervention were already on a favorable trajectory or truly benefited from the program.
- Impacts can be highly heterogenous, with the intervention potentially lowering expenditures for some patients while increasing expenditures for others.
- The data have a hierarchical structure, with repeated observations of patients over time, and with patients clustered in primary care practices.
- The outcome of interest, Medicare spending, is highly variable and skewed. The average patient incurs around $1,000 in costs per month, but some patients incur tens of thousands of dollars in costs.
Partnering with ACIC and the Society for Causal Inference to organize this year’s competition gives Mathematica the opportunity to point some of the brightest minds in causal inference at the challenging evaluation problems we face every day. The results of the competition will advance the field of causal inference and help Mathematica continue to improve the quality of evidence it provides in all its program and policy evaluations.