Small Samples Due to Lower-Than-Planned Enrollment in Impact Evaluations: What to Do?

Small Samples Due to Lower-Than-Planned Enrollment in Impact Evaluations: What to Do?

Evaluation Technical Assistance Brief #5
Published: Sep 10, 2020
Publisher: Princeton, NJ: Mathematica

Authors

Impact evaluations in child welfare and other fields often struggle because of smaller-than-planned sample sizes. Multiple factors might contribute to the problem: The program’s target population might be smaller than was projected, or recruiting and enrolling eligible participants into the study might have proven unexpectedly difficult. Small sample sizes can create difficulties and limitations when estimating the impacts of programs—especially when you had not planned for them during the evaluation design phase. This brief, presented as a series of questions and answers, addresses this specific problem and offers guidance for analyzing data and reporting findings when it occurs.

Efficiency Meets Impact.
That's Progress Together.

To solve their most pressing challenges, organizations turn to Mathematica for deeply integrated expertise. We bring together subject matter and policy experts, data scientists, methodologists, and technologists who work across topics and sectors to help our partners design, improve, and scale evidence-based solutions.

Work With Us