How Much Do the Effects of Education and Training Programs Vary Across Sites? Evidence from Past Multisite Randomized Trials

How Much Do the Effects of Education and Training Programs Vary Across Sites? Evidence from Past Multisite Randomized Trials

Published: Nov 06, 2017
Publisher: Journal of Research on Educational Effectiveness, vol. 10, issue 4
Download
Authors

Michael J. Weiss

Howard S. Bloom

Himani Gupta

Alma E. Vigil

Daniel N. Cullinan

Multisite trials, in which individuals are randomly assigned to alternative treatment arms within sites, offer an excellent opportunity to estimate the cross-site average effect of treatment assignment (intent to treat or ITT) and the amount by which this impact varies across sites. Although both of these statistics are substantively and methodologically important, only the first has been well studied. To help fill this information gap, we estimate the cross-site standard deviation of ITT effects for a broad range of education and workforce development interventions using data from 16 large multisite randomized controlled trials. We use these findings to explore hypotheses about factors that predict the magnitude of cross-site impact variation, and we consider the implications of this variation for the statistical precision of multisite trials.

Follow the Evidence

Interested in the most current findings from Mathematica? Subscribe to our bi-weekly newsletter, Evidence & Insights, to stay up to date with the issues that matter to you.

Sign Me Up