Promoting Opportunity Demonstration: Effectiveness of Reminder Messages for Recruitment

Promoting Opportunity Demonstration: Effectiveness of Reminder Messages for Recruitment

Published: Aug 10, 2020
Publisher: Mathematica

Associated Project

Promoting Opportunity Demonstration (POD) Evaluation

Time frame: 2017–2022

Prepared for:

Social Security Administration

Authors

Heinrich Hock

Michael Levere

David Wittenburg

Background

  • The Social Security Administration (SSA) is carrying out the Promoting Opportunity Demonstration (POD) to evaluate a benefit offset rule for Social Security Disability Insurance (SSDI) beneficiaries. This offset rule reduces benefits by $1 for every $2 earned above the Trial Work Period level, defined as $880 in 2019. It replaces current rules, which are more complex and may result in a sharper loss of all benefits (a “cash cliff”) for beneficiaries who engage in substantial work activity for a sustained amount of time.
  • SSDI beneficiaries in eight states who met certain eligibility requirements could volunteer for POD by responding to a mailing and submitting enrollment materials. The evaluation team encouraged beneficiaries to enroll by sending reminder postcards, among other methods.
  • With SSA’s guidance, the evaluation team began sending an additional round of postcards as a final reminder to further encourage enrollment in POD.

Purpose

  • SSA and the evaluation team used the roll-out of this final reminder postcard as an opportunity to learn more about the effectiveness of reminder messages when conducting outreach to recruit beneficiaries.
  • This brief summarizes findings from an experiment to assess messaging strategies for final reminder postcards sent to 146,548 beneficiaries. The experiment tested whether two components of the postcard affected enrollment. One component was structure: either a fold-over postcard with more details about POD or an open postcard with more generic language about an SSA study. The other component was framing: either an urgent framing that used “act now” language without a deadline or a deadline framing that used “time left” language and stated an explicit cutoff date for enrollment.
  • This brief also describes exploratory findings from a nonexperimental assessment of the final reminder postcards’ overall effectiveness. The evaluation team measured whether the enrollment rate was higher for those sent the postcard relative to a benchmark calculated for a comparison group of beneficiaries who were not sent the postcard.

Findings

  • The structure of the postcard affected enrollment rates; fold-over postcards increased enrollment compared to the open postcards. The framing of the postcard did not affect final enrollment rates, but it might have affected the timing of enrollment.
  • Exploratory results suggest that, overall, the final reminder postcard effort increased the share of beneficiaries who enrolled in POD.
  • These findings highlight the importance of testing messaging strategies and the potential value of repeated contacts when conducting program outreach.

Efficiency Meets Impact.
That's Progress Together.

To solve their most pressing challenges, organizations turn to Mathematica for deeply integrated expertise. We bring together subject matter and policy experts, data scientists, methodologists, and technologists who work across topics and sectors to help our partners design, improve, and scale evidence-based solutions.

Work With Us