Horseshoes, Hand Grenades, and Treatment Effects? Reassessing Bias in Nonexperimental Estimators

Horseshoes, Hand Grenades, and Treatment Effects? Reassessing Bias in Nonexperimental Estimators

Working Paper 16
Published: Mar 30, 2013
Publisher: Oakland, CA: Mathematica Policy Research

Associated Project

Charter Schools: Are They Effective?

Time frame: 2003-2010

Prepared for:

U.S. Department of Education

Authors

Kenneth Fortson

Emma Kopa

Natalya Verbitsky-Savitz

Nonexperimental methods, such as regression modeling or statistical matching, produce unbiased estimates if the underlying assumptions hold, but these assumptions are usually not testable. Most studies testing nonexperimental designs find that they fail to produce unbiased estimates, but these studies have examined weaker evaluation designs. This working paper addresses these limitations and finds the use of baseline data that are strongly predictive of the key outcome measures considerably reduces bias, but might not completely eliminate it.

Efficiency Meets Impact.
That's Progress Together.

To solve their most pressing challenges, organizations turn to Mathematica for deeply integrated expertise. We bring together subject matter and policy experts, data scientists, methodologists, and technologists who work across topics and sectors to help our partners design, improve, and scale evidence-based solutions.

Work With Us