Comparing Job Training Impact Estimates Using Survey and Administrative Data
Download
Associated Project
Workforce Investment Act Adult and Dislocated Worker Programs Gold Standard Evaluation
Prepared for:
U.S. Department of Labor, Employment and Training Administration
The data source used for analysis can affect the accuracy and precision of a study’s estimates. In this paper, we examine three sources used to produce impact estimates for the WIA Adult and Dislocated Worker Programs Gold Standard Evaluation: (1) survey data, (2) administrative data from the National Directory of New Hires (NDNH), and (3) Administrative Tax Data. We further explore the strengths and drawbacks of administrative and survey data by directly comparing estimates produced using the survey data and NDNH. We found that both sources were consistent in their main conclusions about program effectiveness but that impacts calculated using the NDNH were smaller than those calculated using survey data, consistent with previous studies of similar sources. Three factors likely explain most of the observed differences: (1) many survey respondents reported jobs not captured by the NDNH; (2) survey respondents typically overreported earnings in any given job, especially early in the follow-up period; and (3) survey respondents typically underreported the number of jobs they held early in the follow-up period.
More like this from Mathematica
Featured Publication
Evaluation of the Networks for School Improvement (NSI) Initiative
Efficiency Meets Impact.
That's Progress Together.
To solve their most pressing challenges, organizations turn to Mathematica for deeply integrated expertise. We bring together subject matter and policy experts, data scientists, methodologists, and technologists who work across topics and sectors to help our partners design, improve, and scale evidence-based solutions.
Work With Us