Case Study: Improving the Quality of Life for Youth
Nearly 81,000 eligible applicants nationwide were randomly assigned to either a program group allowed to enroll in Job Corps or a control group whose 6,000 members were not allowed to enroll in the program.
The Job Corps evaluation successfully tracked 8 in 10 participants for interviews after four years.
Mathematica analyzed Social Security Administration (SSA) annual summary earnings records (SER) data during the nine years after random assignment.
Notes on Data Collection. The sample was selected from about 81,000 youth who applied to Job Corps for the first time between late 1994 and early 1996. The longitudinal evaluation used a random assignment design—the gold standard in scientific research—to examine the effectiveness of the program. The study—unlike previous evaluations of similar programs conducted in purposively selected sites only—used a nationally representative participant sample. The survey sample included 9,000 program group and 6,000 control group members. Interviews were conducted shortly after random assignment and 12, 30, and 48 months afterwards. Topics included education and training; employment earnings; and non-labor market outcomes such as welfare benefits, crime, alcohol and drug use, health, child care, and mobility. The evaluation included an examination of longer-term earnings impacts using administrative data earnings reported by employers to the Internal Revenue Service (IRS) and SSA and from quarterly wage records reported by employers to state unemployment insurance (UI) agencies. Key evaluation results were based on comparisons between the outcomes of program group members and control group members.
Because Job Corps is one of the most expensive DOL-funded education and training programs in the country, policy makers sought an entity with a proven record of high quality research to evaluate the high-profile Job Corps program. Until Mathematica’s multi-year evaluation, there was no evidence on the effectiveness of the Job Corps program based on a gold-standard experimental design. The evaluation was one of the largest social experimental evaluations ever conducted of an ongoing education and training program.
In 1993, in recognition of our expertise in large-scale, high quality scientific research, DOL’s Employment and Training Administration commissioned Mathematica to conduct a congressionally mandated evaluation of the Job Corps program. The longitudinal evaluation used an experimental design to estimate the effectiveness of the program. Four rounds of survey data and several rounds of administrative earning records were collected to assess long-term program impacts on participants’ employment and related outcomes.
The effectiveness of Job Corps was of particular interest because other federal employment and training programs for low-income youth have not been found effective. Key findings included:
- Job Corps substantially increased the education and training services participants received by about one school year, and the receipt of GED and vocational certificates by about 20 percentage points. It also raised the literacy and numeracy scores measuring skills that adults need to function successfully in daily life.
- Job Corps increased participants’ earnings by about 12 percent in the third and fourth years after random assignment, overall and across most subgroups.
- Job Corps reduced arrest, conviction, and incarceration rates by about 16 percent each.
This case study is for informational purposes only. Mathematica Policy Research, a nonpartisan research firm, provides a full range of research and data collection services, including program evaluation and policy research, survey design and data collection, research assessment and interpretation, and program performance/data management, to improve public well-being. Its clients include federal and state governments, foundations, and private-sector and international organizations. The employee-owned company, with offices in Princeton, N.J.; Ann Arbor, Mich.; Cambridge, Mass.; Chicago, Ill.; Oakland, Calif.; and Washington, D.C., has conducted some of the most important studies of education, disability, health care, family support, employment, nutrition, and early childhood policies and programs.
In 1993, DOL authorized Mathematica to conduct the National Job Corps Study, an independent evaluation of the program’s effectiveness.
Mathematica’s evaluation was based on one of the most rigorous research designs ever used for an ongoing federal program.