What I’ve Learned Evaluating Large-Scale Agriculture Programs—And What We Can Do About It

What I’ve Learned Evaluating Large-Scale Agriculture Programs—And What We Can Do About It

Feb 10, 2021
Woman in field of grains

This blog is part of our Growing the Evidence series, which contains evidence and insights uncovered by Mathematica’s researchers in the international food and agriculture sector.

Experts agree that low-income farmers worldwide need technology, training, and financing to achieve food security, lift themselves out of poverty, and build resilience to climate change. But how to structure, combine, and deploy these supports is the multibillion-dollar question. To uncover insights about what works, I’ve spent more than 10 years evaluating large-scale farmer training programs in Latin America and sub-Saharan Africa. Here are three big takeaways from my research:

Improving farmers’ well-being through large-scale assistance is a difficult task.

Aggressive targets for number of farmers trained often dilute large programs to standardized and light-touch assistance. Commonly, diluted assistance fails to alter farmers’ practices and negotiating power within complex value chains, resulting in unchanged income and consumption patterns. Further complicating matters, farmers face variable weather patterns and the constant threat of crop disease. In this competitive and precarious context, agriculture programs must be nearly flawlessly designed and executed to have a lasting impact on low-income farmers. Perhaps the most difficult piece is adapting scientifically proven models to new farmer groups and settings without losing the original winning formula. These adjustments are extremely difficult to make with 500 farmers, much less 50,000.

Successful programs structure assistance according to farmers’ most pressing needs.

Although it’s very hard to get agriculture programs right, some programs are successful in putting farmers on a path toward food security and prosperity. What these successful programs have in common is a comprehensive understanding of farmers’ binding constraints to growth. These include deficits in farmers’ skills, suboptimal access to inputs and markets, and incentives and psychological factors that hinder advancement. Leveraging this understanding at baseline, successful programs attack farmers’ binding constraints head-on and with enough resources to make a difference. For example, the Millennium Challenge Corporation funded a program in El Salvador that provided dairy farmers with machinery, subsidized inputs, and dozens of hours of tailored assistance over several months. This robust assistance successfully addressed production bottlenecks and skills gaps, empowering farmers to increase annual farm income by more than $1,800, on average. With a per-participant cost of more than $3,000, this program was certainly expensive. But if farmers’ gains persist over several years, its benefits will far exceed its costs.

Farmers with more advantages from the start often drive positive results.

In our evaluations, we often see different results across different segments of farmers—even when assistance is pretty uniform. Farmers with relatively high incomes tend to leverage their access to financing and irrigation to take full advantage of training, often making dramatic gains in production and sales in just one season. In contrast, subsistence farmers and women have more difficulty improving their situations through training, given the multitude of constraints that they face. As a result, we have found income gains concentrated among the top quartile of farmers, with more than half of farmers no better off than when the program began. These findings highlight a natural tension in the sector between helping the poorest of the poor reach food security, on one hand, and helping more established producers formalize their operations, on the other.

All of us—including practitioners, donors, and researchers—can learn from these takeaways to better serve the world’s poor. Here are some initial thoughts on how we can move large-scale agriculture programming forward:

Practitioners should listen harder, identify natural segments, and start small.

Taking a lesson from the most effective programs, practitioners should build serious baseline stakeholder consultations and community assessments into their work. These consultations should focus on understanding farmers’ binding constraints to growth, which serve as the starting point for any effective program. Whenever feasible, practitioners should organize farmers into segments, and provide each segment with the right mix of supports to unlock its constraints. And because the sector presents so many complexities and risks, practitioners should start simple. In particular, they should use small-scale pilots to arrive at the best combination of supports and the most appropriate dosage before scaling. (Scaling-up is hard enough as it is, given institutional factors, political challenges, and potential equilibrium effects. Why further complicate matters?)

Researchers should do more to support design and program improvement.

Researchers need to play a much larger role in program design. We need to distill our own evaluation findings and the larger evidence base to help strengthen the next generation of programs. We should also improve how we facilitate learning sessions with policymakers and practitioners to help them infuse new programs with best practices. Finally, we should do more quick-turnaround formative work to help refine pilots before they are scaled. These early research investments will increase the impact of programs down the road.

Donors should incentivize quality programming and tangible results.

Because they hold the purse strings, donors must drive the changes I suggest above for practitioners and researchers. First, if at all feasible, donors should make longer commitments (of five years or more) to geographies and farmer groups. These commitments would give practitioners enough time to complete proper assessments and improve programs through pilots. Donors should also resist the urge to set aggressive training targets, at least in the first couple of years as the program is taking shape. They should also incentivize tangible changes in farmer income and consumption, potentially with results-based financing arrangements. Lastly, donors should commission more research focused on crafting evidence-based designs and improving programs in real time. The field’s current focus on rigorous retrospective evaluation does much to advance the evidence base, but little to help the programs being studied.

In my experience, there is no simple answer to the multibillion-dollar question of what works in agriculture programming. There are actually thousands of correct answers—one for each unique segment of farmers in each unique context. Fortunately, the road map to finding what works in agriculture (or any sector, really) is pretty universal. Inevitably, it involves a comprehensive review of the latest evidence, careful design to address stakeholders’ binding constraints, and a lot of trial and error in refining supports. It also requires all of us to have the humility and patience to get programs right before entertaining notions of replication and scale-up.

About the Author