Advancing Educational Equity: Tools for Evidence Building

In partnership with the Bill & Melinda Gates Foundation, Mathematica developed a set of measurement and evaluation (M&E) tools to help education grantees generate evidence about their solutions (such as programs, practices, or products) to address education’s most persistent problems. To expand the reach of this work and promote the adoption of evidence-based solutions more broadly, we adapted the tools for use in many fields and by a diverse set of users—organizations, funders, research partners, and other interested parties. Keep reading to learn more about these tools and the M&E process or click on an icon below to jump to a step in the process. You can also download the complete toolkit.

 

What is the Measurement and Evaluation process?

Mathematica’s suite of measurement and evaluation (M&E) tools provides a road map for generating timely and actionable evidence about what works for whom, and in what context. Whether you are a developer refining evidence-based solutions, a funder exploring a great new idea, or a researcher, you can use these tools to promote rapid innovation and scaling of promising solutions.

Each step of the M&E process has tools to help guide you to a deeper understanding of your solution. There are built-in checkpoints to encourage users to pause and reflect on learning and to refine the solution and the M&E plan as needed.

Additionally, equity and community voice are a core principle embedded in the M&E process. Organizations should meaningfully engage communities across all steps to ensure the solution builds on the strengths and assets and meets the needs of the community they will serve.

For definitions of terms used in the tools, please refer to this glossary.

Step 1: Assess Measurement and Evaluation Needs and Interests

These tools introduce the iterative approach to evidence building and help organizations articulate their M&E goals, interests, and needs.


Kids in a math class

Introducing the evidence-building pipeline and development framework

Play Video

Right-Sizing Research to Inform Design and Decision-Making, Part 1

In this video, Mathematica experts introduce an iterative approach to evidence building that they created to help organizations design, refine, and test solutions that seek to address educational and other social challenges.

Expand to See Tools for Step 1
result(s) for are listed below
Select View
Summary

The Measurement and Evaluation Needs Inventory is a tool designed to assist an organization in conducting a self-assessment of its existing measurement and evaluation capacity, with the goal of identifying priority areas for support from an external research partner.

Download
Summary

The Prior Research Table is a tool designed to help organizations document previous research on solutions to inform future evaluations.

Download

Step 2: Plan Measurement and Evaluation

These tools help organizations develop their own measurement and evaluation plan. The checklists in particular provide guidance on working through the evidence-building phases.

Expand to See Tools for Step 2
Filter
Hide
result(s) for are listed below
Select View
Summary

This resource provides educators, school district leaders, researchers, and funders a set of measures and approaches to assess the extent that math instruction and interventions change what students know and how they feel about math, as well as how they use math outside of the classroom.

Download
Summary

This menu describes a set of teacher- and student-level measures and approaches to understand the extent that writing instruction and interventions change how teachers think about and teach students about writing, and what students know and how they feel about writing.

Download
Summary

The Measurement and Evaluation Template is a road map organizations and funders can use to develop a detailed measurement and evaluation plan with support from a research partner.

Download
Summary

This template is for organizations that want to incorporate the needs and preferences of the communities they intend to reach into their program, practice, or product’s design. It includes space for organizations to document a description of their program, practice, or product; their theory of change; their learning objectives and approach; and a timeline to carry out the work.

Download
Summary

Phase 1 of the Measurement and Evaluation Checklist guides users through the process of co-designing a solution with a partner and developing a well-defined theory of change for how that solution is expected to improve outcomes for students in a priority community.

Download
Summary

Phase 2 of the Measurement and Evaluation Checklist guides users through the steps to refine a solution based on lessons learned during implementation. By following these steps, users will develop descriptive evidence that the refined solution was successfully implemented in a priority community.

Download
Summary

Phase 3 of the Measurement and Evaluation Checklist guides users through the steps to conduct a small pilot study and demonstrate that a solution is associated with improved outcomes for students in a priority community.

Download
Summary

Phase 4 of the Measurement and Evaluation Checklist guides users through the steps to conduct a large pilot study and demonstrate with a high degree of confidence that a solution leads to improved outcomes for students in a priority community.

Download
Summary

This guide is designed for organizations implementing solutions, with support from a research partner. It provides guidance on when and how to measure usability, usefulness, and utilization—key indicators of success during early implementation.

Download
Summary

This guide is designed to support organizations at different stages of solution development and implementation define a set of targets for each research question they will answer. Targets enable organizations or their funders to define success.

Download
Summary

This guide is designed for funders and organizations implementing solutions with support from a research partner. It provides an overview of the sample size considerations for each of the evidence-building phases.

Download
Summary

This tool offers strategies for engaging community members as part of a research team and including their experiences, perspectives, and expertise throughout the research process.

Download

Step 3: Conduct Measurement and Evaluation

These tools assist organizations executing their M&E plan, including collecting the data needed to address each research question.


Rightsizing Research to Inform Design and Decision Making, Part 2

In this video, Mathematica experts introduce an iterative approach to evidence building that they created to help organizations design, refine, and test solutions that seek to address educational and other social challenges.

Two middle school girls in class

A Case Study

Play Video
 

Rightsizing Research to Inform Design and Decision Making, Part 3

In this video, Mathematica experts provide a deep dive into each of the five measurement and evaluation criteria that are part of an evidence-building pipeline they developed to support organizations, research partners, and funders develop, refine, and test solutions.

Two middle school boys in class

Measurement and Evaluation Criteria

Play Video

Step 4: Analyze and Report Results

These tools guide organizations through the final step of the M&E process, which includes analyzing collected data, determining next steps, and reporting the findings.

Expand to See Tools for Step 4
Filter
Hide
result(s) for are listed below
Select View
Summary

This resource is intended to guide an organization’s approach to data interpretation, with the goal of interpreting study data collaboratively with members of the community or communities where research is taking place.

Download
Summary

Organizations, research partners, and funders can use the Measurement and Evaluation Reporting Template to summarize and share findings from a study.

Download
Summary

This tool offers strategies for integrating community perspectives when disseminating research. It outlines key steps for co-interpretation—or collaborating with community members to interpret data—along with strategies for sharing findings that are accessible to and reach communities.

Download

Related Staff

To connect with one of our experts on this project or to get more information about these tools, please send an email to info@mathematica-mpr.com.

Kate Place

Kate Place

Senior Researcher

View Bio Page
Virginia Knechtel

Virginia Knechtel

Director

View Bio Page
Kara Conroy

Kara Conroy

Senior Researcher

View Bio Page
Megan Shoji

Megan Shoji

Principal Researcher

View Bio Page
Ryan Ruggiero

Ryan Ruggiero

Researcher

View Bio Page
Alexandra  Resch

Alexandra Resch

Principal Researcher

View Bio Page
Gregory Chojnacki

Gregory Chojnacki

Senior Researcher

View Bio Page
Lindsay Fox

Lindsay Fox

Senior Researcher

View Bio Page
Kaleen Healey

Kaleen Healey

Senior Researcher

View Bio Page
George Smith

George Smith

Researcher

View Bio Page
Marykate  Zukiewicz

Marykate Zukiewicz

Senior Researcher

View Bio Page
Francesca Venezia

Francesca Venezia

Researcher

View Bio Page