Scaling qualitative analysis with generative AI

Analyzing complex qualitative evidence across three countries

For a retrospective evaluation of irrigation investments, the Gates Foundation sought to understand why farmers in Kenya, Ethiopia, and Nigeria continued or stopped using the irrigation tools and practices supported by the foundation.

The study generated a large number of interview and focus group transcripts that required consistent thematic analysis and clear links between every finding and its supporting evidence. The project team needed a more scalable approach to accelerate the analysis without compromising the quality or traceability of the findings.

Combining generative AI with a structured thematic coding process

Mathematica deployed its in-house, Python-based thematic coding library, which relies on generative AI models through Amazon Bedrock, to analyze the transcripts using project-specific coding guidance. The tool identified relevant themes, extracted supporting quotations, and linked each excerpt to its source, enabling the researchers to validate results and focus on interpretation rather than initial coding.

Faster coding, greater analytical capacity, and stronger traceability

The AI-enabled process greatly reduced the effort needed to analyze a large qualitative data set.

• Faster thematic analysis: The project team used AI-enabled thematic coding to analyze approximately 80 interview and focus group transcripts. While a comparable manual coding effort could have required more than 20 working days and a team of three or more coders, the AI-supported approach was carried out primarily by one researcher and one programmer, who together spent an estimated 4 to 6 person-days implementing and executing the analysis.
• Expanded analytical capacity: Researchers spent less time coding transcripts and more time comparing findings across countries, identifying patterns, and developing insights.
• Improved consistency and confidence: Using a common thematic framework helped ensure the team was coding similar evidence consistently across transcripts. Every quotation remained linked to its original interview or focus group discussion, making it easy for researchers to verify findings.