What Works for Whom? A Bayesian Approach to Channeling Big Data Streams for Public Program Evaluation
Download
Key Findings
- The novel Bayesian approach presented in this article produces better inference than the standard approach, and does so both more efficiently and without sacrificing crucial methodological principles.
- Bayesian adaptive design benefits study subjects by assigning them to more effective treatment arms.
- Bayesian adaptive design benefits evaluators performing the study by allowing for smaller and more informative studies.
- The benefit to study subjects may be attractive to practitioners who often have concerns about the ethical implications of denying services or staying with a treatment that is not effective.
In the coming years, public programs will capture even more and richer data than they do now, including data from web-based tools used by participants in employment services, from tablet-based educational curricula, and from electronic health records for Medicaid beneficiaries. Program evaluators seeking to take full advantage of these data streams will require novel statistical methods, such as Bayesian approach. A Bayesian approach to randomized program evaluations efficiently identifies what works for whom. The Bayesian approach design adapts to accumulating evidence: Over the course of an evaluation, more study subjects are allocated to treatment arms that are more promising, given the specific subgroup from which each subject comes. We identify conditions under which there is more than a 90% chance that inference from the Bayesian adaptive design is superior to inference from a standard design, using less than one third the sample size.
More like this from Mathematica
Featured Publication
Opioid Use Among Social Security Disability Insurance Applicants
How do you apply evidence?
Take our quick four-question survey to help us curate evidence and insights that serve you.
Take our survey