Tips for Planning an Impact Evaluation

Tips for Planning an Impact Evaluation

Evaluation Technical Assistance Brief Number 3
Published: Dec 01, 2017
Publisher: Washington, DC: Mathematica

Authors

Robert Santillano

Debra Strong

The Regional Partnership Grant (RPG) program supports partnerships between providers of child welfare services, substance abuse disorder (SUD) treatment, and other social services that enhance the safety and well-being of children who are in, or at risk of, out-of-home placement because of a parent’s or caretaker’s SUD. The Children’s Bureau in the Administration for Children and Families directs the RPG program, which is a key federal program that the United States Congress authorizes to solely and specifically address the issues that arise at the intersection of child welfare and substance use.

The Children’s Bureau funded this brief for groups that receive RPG grants or other grants and want to conduct an evaluation that can identify the causal impacts of a program designed for families involved with—or at risk of involvement with—the child welfare system. It draws on the experiences of previous cohorts of RPG grantees to give tips about how to plan an impact evaluation. The tips focus on impact evaluation designs in which groups were formed either through a randomized controlled trial (RCT) or a quasi-experimental design (QED). The discussion of these two strategies highlights pitfalls and proposed solutions to them.

Efficiency Meets Impact.
That's Progress Together.

To solve their most pressing challenges, organizations turn to Mathematica for deeply integrated expertise. We bring together subject matter and policy experts, data scientists, methodologists, and technologists who work across topics and sectors to help our partners design, improve, and scale evidence-based solutions.

Work With Us