Analyzing Grouped Administrative Data for RCTs Using Design-Based Methods

Analyzing Grouped Administrative Data for RCTs Using Design-Based Methods

Published: Feb 01, 2020
Publisher: Journal of Educational and Behavioral Statistics, vol. 45, issue 1 (subscription required)

Peter Z. Schochet

This article discusses estimation of average treatment effects for randomized controlled trials (RCTs) using grouped administrative data to help improve data access. The focus is on design-based estimators, derived using the building blocks of experiments, that are conducive to grouped data for a wide range of RCT designs, including clustered and blocked designs, and models with weights and covariates. Because of the linearity of the regression model underlying RCTs, the asymptotic properties of design-based estimators using group-level averages—formed randomly or by covariates for nonclustered designs and as cluster-level averages for clustered designs—match those using individual data. Furthermore, design effects from aggregation are tolerable with moderate numbers of groups and few covariates, suggesting little information is lost in these cases. Ecological inference methods for subgroup analyses, however, yield large design effects. Several empirical examples using real-world education RCT data demonstrate the theory.

How do you apply evidence?

Take our quick four-question survey to help us curate evidence and insights that serve you.

Take our survey