Using Video Recordings as a Data Collection Strategy: Lessons from the Evaluation of Employment Coaching
OPRE Report #2021-226
Download
Clients
Key Findings
- Overall, the study team found that collecting and analyzing video data for employment programs is feasible and can be a useful addition to implementation studies.
- Collecting video data with consent from participants can help evaluation teams understand the implementation of interventions that rely on staff—participant interactions.
- Researchers should be sure to video enough staff—participant interactions to support conclusions.
- In addition to using a sampling plan, monitoring how the videos are collected and having multilingual coders can help ensure the recording sample is representative.
- Tailoring the coding form to best capture the unique elements of each intervention being studied and using highly trained coders is important when observing multiple interventions with nuanced program elements.
Implementation studies of employment programs typically involve interviews with managers, frontline staff, and partners, and sometimes interviews or focus groups with participants. Although evaluation teams might view some activities, observations of interactions between staff and participants are often limited. Yet for many programs, these interactions are central to the intervention.
This brief discusses the use of video recordings of interactions between program staff and participants as a data source for an implementation study of four employment coaching interventions for people with low incomes. The study team piloted this data collection as part of the Evaluation of Employment Coaching for TANF and Related Populations. The brief describes the process for video recording coaching sessions and analyzing the videos, provides examples of what was learned from the data, and offers lessons for researchers considering using video recordings as a data source in future studies.
More like this from Mathematica
Featured Publication
Employment Coaching During the COVID-19 Pandemic: Lessons Learned
Efficiency Meets Impact.
That's Progress Together.
To solve their most pressing challenges, organizations turn to Mathematica for deeply integrated expertise. We bring together subject matter and policy experts, data scientists, methodologists, and technologists who work across topics and sectors to help our partners design, improve, and scale evidence-based solutions.
Work With Us