Creating programs to help young people with disabilities successfully transition from school to work is an important policy concern.
Programs targeting teenagers and young adults are more likely to be effective than those initiated at later stages, when individuals' expectations about disability and dependence are more entrenched. In addition, interventions that reduce average lifetime duration on the disability rolls could generate substantial savings for the federal government.
In September 2005, the Social Security Administration (SSA) awarded Mathematica a nine-year, $47 million contract to evaluate the Youth Transition Demonstration (YTD) projects, designed to help youth with disabilities maximize their economic self-sufficiency as they move from school to work. To accomplish this goal, the projects provided benefits counseling, career counseling, job development, job placement, and services to support continued employment. A key element of the initiative was the waiving of certain SSA disability program rules to provide enhanced financial incentives for youth with disabilities to initiate work or increase their work activity. Ten YTD projects were implemented across the country, of which six were selected to participate in Mathematica’s random assignment evaluation. Those projects were located in Bronx County, New York; Colorado (4 counties); Erie County, New York; Miami-Dade County, Florida; Montgomery County, Maryland; and West Virginia (19 counties). All of the YTD projects had completed the provision of services to participants by March 2012.
The YTD projects worked with youth ages 14 to 25 who were receiving Supplemental Security Income, Social Security Disability Insurance, or Childhood Disability Benefits, and those who were at high risk of receiving these benefits in the future. Although the projects were required to conform to a broad design for the interventions, they had considerable flexibility regarding the subgroups of youth they served and the services and supports they provided.
Mathematica assisted the projects with their interventions and led the evaluation of their effectiveness in reducing the likelihood that participants will become lifelong recipients of disability benefits. Working with its subcontractors, MDRC and TransCen, Inc., Mathematica developed intervention strategies, selected six sponsoring organizations to implement YTD projects in a multiyear demonstration, and assisted the sponsors in running their projects. The evaluation was based on an experimental design. Outcomes for youth were measured through administrative data sources and surveys conducted 12 and 36 months after they enrolled in the evaluation and were randomly assigned to a treatment group, which was eligible to receive the SSA waivers and project services, or a control group, which was subject to the standard rules for SSA disability programs and could receive only those services that were available independently of the demonstration projects.
TransCen provided the sponsoring organizations with intensive technical assistance in designing their YTD projects and implementing them on a day-to-day basis. This assistance entailed close contact with project staff over a period of three years. It covered a range of topics, but the greatest effort was devoted to helping project staff reach out to employers and place participants in jobs.
Key components of the evaluation included a process study of project implementation, an impact study of the interventions for youth with disabilities, and a project cost analysis. Interim reports on each of the six random assignment YTD projects were completed in 2011-2012. These reports present findings from the process study and from the analysis of impacts during the first year after youth enrolled in the evaluation. A comprehensive final report was completed in November 2014. It presents estimates of the impacts of the projects during the third year following enrollment and findings form the cost analysis.
Data Collection Highlights
Mathematica survey staff worked from lists of SSA beneficiaries and with the projects directly to locate youth with disabilities, determine their interest in enrolling, and obtain their consent for the research. We used special techniques to engage youth and collect accurate and complete baseline data from them. These techniques included developing materials and forms at a sixth-grade reading level, avoiding acquiescence bias and undue respondent burden, and allaying concerns of protective parents and gatekeepers. Our efforts were notably successful in convincing youth to participate—20 percent of those who were eligible enrolled in the study, compared with only 2 to 5 percent in similar evaluations in the past. In all, the study includes 5,103 youth.
Our survey team conducted baseline interviewing by telephone and collected consent forms by mail with telephone reminders and in-person collection when necessary. We conducted mixed-mode 12-month and 36-month follow-up interviews by telephone with field follow-up. The response rates among all youth who enrolled in the evaluation and were not deceased at the time of the follow-up surveys were 87 percent for the 12-month survey and 82 percent for the 36-month survey.
Highlights from the Interim Reports
The six project-specific interim reports present findings from the YTD evaluation’s process study and the analysis of one-year impacts. These reports show that the intensity of services and focus on employment were high for four of the projects. Three of those projects had positive and statistically significant impacts on paid employment during the year following random assignment and two of them also had significant positive impacts on annual earnings. The one-year impacts were largest for the project in West Virginia, where treatment youth were employed at a 43 percent rate and had average earnings of $1,559 during the initial follow-up year. If those youth had not been given the opportunity to participate in the project, we estimate that only 24 percent would have been employed and their average earnings would have been just $1,035. We found no impacts on employment and earnings for the two projects in which the intensity of services and focus on employment were low.
Highlights from the Final Report
The final report on the YTD evaluation presents project-specific findings from the evaluation’s analysis of three-year impacts. Three of the six projects had positive and statistically significant impacts on the proportion of youth who were employed for pay at any time during the third year after they enrolled in the evaluation. Those impacts were on the order of 7 percentage points, meaning that the employment rate was 7 percentage points higher for treatment group than for control group youth. Two of the projects significantly reduced youth contact with the justice system. The YTD project in Miami-Dade County, Florida, had statistically significant impacts on the greatest number of year-three outcome measures: it increased paid employment, earnings, and total income and decreased contact with the justice system. An April 2015 Mathematica issue brief, “Three-Year Impacts of Services and Work Incentive on Youth with Disabilities,” summarizes the findings presented in the evaluation’s final report.
