In March, when the Biden administration released its budget request for fiscal year 2025, it not only offered a blueprint for the president’s policymaking agenda—it was also the latest indication of how the White House and federal agencies are going beyond evidence generation to use evidence as a guide in making program investments that can improve Americans’ lives. For this episode of Mathematica’s On the Evidence podcast, Nichole Dunn, the vice president of federal policy at Results for America, joins the show to discuss evidence-based policy in the president’s budget as well as larger trends in federal and state funding of evidence-based initiatives and programs.
On the episode, Dunn speaks with Mike Burns, senior director of communications and public affairs at Mathematica, about American Rescue Plan spending, the growth of evidence clearinghouses, the increasing level of federal investment in program evaluation, the implications of the presidential election for federal investment in evidence-based policy, the implementation of the Foundations for Evidence-Based Policymaking Act, the recent formation of the bipartisan Congressional What Works Caucus, the Congressional Evidence-Based Policymaking Resolution, and the potential for evidence-based policy to bypass partisan gridlock in Washington.
“When we talk about government these days, it's usually about the gridlock and political divisions in Congress, but there's a whole other side to the story that people don't hear,” Dunn tells On the Evidence, echoing comments made by the chief executive officer and co-founder of Results for America, Michele Jolin, in an op-ed in March. “Across the country, from places like Minnesota to D.C., there's been really a revolution, I would say, about how taxpayer dollars are spent to get better results.”
Watch below.
View transcript
[NICHOLE DUNN]
When we talk about government these days, it's usually about the gridlock and political divisions in Congress, but there's a whole other side to the story that people don't hear. Across the country, from places like Minnesota to DC, there's been really a revolution, I would say, about how taxpayer dollars are spent to get better results.
[J.B. WOGAN]
I'm J.B. Wogan from Mathematica, and welcome back to On the Evidence. This episode is part of our ongoing Evidence in Government series where we talk about new developments in the halls of government and the role that evidence can or should play in decisions that could improve people's lives. My colleague, Mike Burns, leads these episodes. Mike previously worked in Public Affairs for the U.S. Department of Housing and Urban Development, the U.S. House Appropriations Committee, and for several members of Congress.
On this episode, Mike talks with Nichole Dunn about the Biden administration's fiscal year 2025 Budget. Nichole is the Vice President for Federal Policy at the nonprofit Results for America. Their conversation revolves around the inclusion of evidence in public policy, not only in this budget but in the legislative branch and at the state and local levels of government. I hope you enjoy the episode.
[MIKE BURNS]
Thank you for joining us, Nichole. I think this is the second year we -- we've spoken about the President's budget request, so definitely glad -- glad to have you back and to get your insights.
[NICHOLE DUNN]
I can’t believe it’s been a year.
[MIKE BURNS]
This is the President's final budget request of his term, and perhaps his presidency. Does that impact how you approach reviewing the budget and what you're looking for in terms of evidence and data? How do the spending caps from the debt ceiling agreement factor in?
[NICHOLE DUNN]
Yeah. Thanks. Ah, no -- well, we've worked really well with the Biden administration on promoting our evidence agenda. We've worked with presidents from both parties to maintain and increase evidence priorities, so there really isn't a change in how our RFA approach our work, depending on a change in administration. So it's been across Democratic and Republican administrations that the federal government has really made great strides in moving more dollars to programs with a track record of effectiveness. So -- and with regard to the spending caps in the debt ceiling agreement, while they pose some challenges, our approach is really to ensure that the federal dollars being spent, regardless of the levels, are going to programs with evidence of effectiveness and a proven track record.
[MIKE BURNS]
So it's interesting you raise the point about investments in programs with a proven track record because I wanted to ask you -- since you were on the podcast last year, what has changed or developed in the evidence ecosystem? And how does -- assuming it does, how does the President's budget requests for FY25 reflect those developments?
[NICHOLE DUNN]
Yeah, that's a great question. There has been so much movement in the field since the last time we talked. One of the things that I've really been thinking about is this growing conversation about moving from evidence generation to evidence use. So how are we using the data that we have, the evidence we've developed, to change how we're spending dollars. So, you know, when we talk about government these days, it's usually about the gridlock and political divisions in Congress, but there's a whole other side to the story that people don't hear. Across the country, from places like Minnesota to DC, there's been really a revolution, I would say, about how taxpayer dollars are spent to get better results. So in a report that we released earlier this January written by Harvard's Dr. Christina Eller, highlights this progress.
So back in 2013, government only had one evidence clearinghouse, but now there's over ten of them with more than 20,000 proven programs. But it's not just about having that info. We need to use it. We need to change how money is getting allocated. And -- and since 2013, there has been a huge increase in funding for programs backed by evidence, like Maternal, Infant, and Child Home Visiting program. And in places like Minnesota, thousands of families are benefitting from home visits thanks to this funding. So I think that conversation about evidence use versus evidence generation has been one of the really important developments. A couple others I'll just quickly mention. It's been really great to see the impact of the use of data and evidence in the implementation of the recovery programs like the American Rescue Plan and the Bipartisan Infrastructure Law. There's really been an emphasis on funding evidence-based programs, building evaluation into program design, and really learning and quickly using that learning.
And you can see some of those great examples on the American Rescue Plan data and evidence dashboard that we did together with Mathematica. A couple other -- big efforts over the last year are trying to simplify grants to make them more accessible to grantees, simpler to apply for, simpler to manage once a grantee has received the funding. And then, finally, I would be remiss not to mention that today the White House Office of Management and Budget will release their new uniform grant guidance. This affects how over $1.2 trillion in annual federal financial assistance to state, local, and tribal governments is awarded and spent. So this is exciting in our wonky data and evidence world. The new guidance simplifies language and reduces red tape.
And, when fully implemented, really has the potential to make federal grants more accessible and encourage state and local governments to invest funds in data infrastructure and evidence building, including an integrated data system so we can track outcomes and -- and determine if we're improving the lives of people that our government serves. So -- Mike, I didn't answer the question yet about how this relates to the President's budget. So a little bit on that. I think that, you know, we had some key recommendations in the report I mentioned earlier by Dr. Eller about fostering demand for data and evidence and setting aside for federal funds.
And, really, funding the full implementation of the Evidence Act. And the budget supports a lot of those recommendations, including hiring more program evaluation experts, maintaining authority for the Department of Labor to allocate funds for program evaluation, and giving leaders tools and training to understand how evidence can shape policies. So some of these initiatives really led by the evidence team at OMB are the OMB Federal Evaluation Toolkit and the Evidence and Evaluation Community of Practice Workshops. And, you know, again going back to the recovery programs, we've seen evaluation incorporated into new initiatives such as the CHIPS Incentives Program and expanded Medicaid maternal health services.
So there's a big push for better data resources, beefing up the Census Bureau surveys, and improving workforce data. So, overall, I think there were a lot of great efforts focused on making sure the government's decisions are backed by solid evidence.
[MIKE BURNS]
So I actually joined that event. I was able to -- to watch it. I thought it was great. Dr. Eller is a friend of the podcast. We've had the OMB Evidence Team on. It's always great to see all these synergies. Was there anything -- you highlighted some of the major takeaways from that event, from that research. Was there anything that surprised you that came out of it?
[NICHOLE DUNN]
You know, one thing that -- I guess it wasn't a surprise but it was really great to have was the survey data from federal agency leaders. So I just talked a lot about the great things in the President's budget, but I would say that one thing that came out of the report is a -- a need for -- for more support for -- federal agency leaders doing this work. You know, I'm sure Dr. Eller may have mentioned this when you chatted with her, but these survey results were very striking. So, based on limited resources, 11 of the 17 federal agencies interviewed indicated that they can spend less than one million dollars on significant evaluations.
And ten of the 17 said that they will not be able to meet at least a portion of their current learning agenda. Eleven of the 17 evaluation officers recorded that they spend less than three-quarters of their time actually on their evaluation officer job because they're allocated to other roles and -- and that take up a lot of their time. So I think one of the things that came out of the report was around the need to elevate these roles and make sure that the folks generating evidence are in the policy conversations and have a seat at the policy making table so that we can really move to that evidence use. So I think that is one thing that came up over and over again is that we really need to move from evidence generation.
While still important and RFA will continue to advocate for more funding for evaluation and building new information about what works for who under what circumstances, but we need to use that information, and we need to use it in our -- in our standard government processes. Let's get it in grants. Let's get it in our budget decision making. Let's use it in federal legislation. So I think that was one of the real keys that came out of the report.
[MIKE BURNS]
And speaking of -- of requests for more funding, we're seeing budget season is underway. Cabinet secretaries testifying on their FY25 requests. What is one thing you'd want to convey to them? Or you would want them to convey, I should say, about investing in evidence and data. And, conversely, if you're a member of Congress, what's one question you would want to ask about the budget requests along those lines?
[NICHOLE DUNN]
Yeah. I think I would, you know, ask to see the evaluation reports. Ask to see -- how are you defining evidence and prioritizing it in your federal grants? I think, you know, going back to the report again, it had ten examples of where focusing on programs with evidence of effectiveness made impacts for family and children across the country. It is, you know, the one -- one of the things that has consistently worked over and over to deliver on economic mobility outcomes.
So I would ask, "What does the evidence say? And how are you using that evidence in decision making?" And that's how RFA is focusing our requests for the FY25 to Appropriations Committee. We are asking for additional funding for programs that both define and prioritize evidence in their competitive grants and have shown to be effective. For example, the Postsecondary Success Grants, the Promise Neighborhoods were requesting, you know, doubling that investment because of its effectiveness. Full Service Community Schools, and workforce development programs like re-entry employment opportunity programs and programs that improve third grade reading like the Comprehensive Literacy Development Grant.
So really focusing on programs with evidence of effectiveness. And then thinking about report language. How can Congress direct agencies to do things like prioritize evidence in their federal grants? Work with OMB around the new uniform guidance to ensure grantees understand that grant funds can be spent on data and evaluation. So we're really focused in those areas.
[MIKE BURNS]
This past November, Results for America released its 2023 Invest in What Works -- State Standard of Excellence Report, showcasing efforts across 46 states to build and use evidence and data. Did you see any examples in the budget request where the administration is learning from states or adopting successful efforts at the state level?
[NICHOLE DUNN]
Yeah. I think we have, you know, seen two things. One, federal policy matters and can incentivize state and local governments in -- to move in specific directions. So, for example, in the Americorp State and National Grants, they prioritized data and evidence in their application. So 64% of their -- their -- those grant dollars go to evidence of effectiveness. And we've seen states replicate those applications at the state level. But one of the things that we've seen, I think the federal government can learn from what states are doing. They -- I think federal agencies could look to the best practices in evidence-based budgeting that are being implemented in states.
So the 2023 Invest in What Works -- Standard of Excellence featured ten states that have put outcomes for the people they serve front and center by defining and prioritizing evidence of effectiveness in state budgets. So Colorado, Illinois, Minnesota, North Carolina, and North -- New Mexico, Oregon, Rhode Island, Tennessee. So really existing to see these states move in this direction, and we'd really like to see, you know, a strong framework for agencies to define and prioritize evidence in their budget requests.
But, you know, some of the other programs I mentioned earlier, we learn about the evaluations of effectiveness from implementation at the state level, like Postsecondary Success Grants. We're seeing those implemented, and that's how we learn, you know, about their effectiveness is funding evaluations of the implementation of those programs. And then we see them receive increased funding at the federal level. So it's a -- a good cycle of learning.
[J.B. WOGAN]
I apologize if this is covered in Dr. Eller's report and I missed it, but I know that the federal standard of excellence that RFA puts out has some tracking about agencies that invest at least a -- 1% of their program budget in evaluation-related activities. And I -- I -- I saw that that's been coming out since 2013. Are there any trends that are worth noting in terms of the number of agencies or which agencies are actually making that -- that level of commitment to evaluation in their budgets?
[NICHOLE DUNN]
Yeah. I -- there is. It has -- it's been an increasing trend. And we did see some -- we've had some exciting set-asides like the set -- AGA Higher Education set aside for evaluation we were thrilled about last and it was continued this year. So I think it is a trend forward. Where agencies have struggled, it really is through limitations through Congress. So we need to continue to work with Congress to make sure that agencies have the flexibility to set aside those funds for evaluation.
[MIKE BURNS]
So I want to get a little -- get a little wonky here. I -- I was looking through my favorite --
[NICHOLE DUNN]
Can we get more wonky?
[MIKE BURNS]
-- we -- I think we can. We were going through -- I was going through my -- my favorite part of the budget, perhaps -- the analytical perspectives chapter. And the chapter mentions or it looks at, you know, whether areas and programs as part of the implementation of the American Rescue Plan had their intended outcomes. I started to think about the recent analysis that RFA and Mathematica conducted on What Works Cities. And that looked at, you know, how well states are using data and evidence in investing their ARP state and local fiscal recovery funds. Can you tell us a little bit more about this study and its implications?
[NICHOLE DUNN]
Yeah. No, this is really exciting, and I think it's a great point to just make about the learning agendas, particularly the -- our learning agenda. There was a very strong effort by the administration to make sure, as we implemented these programs, there was evaluation built in in the beginning and that we're -- we're learning a lot about what has worked. But specific to the report built with Mathematica, the report built on our previous ARP data and evidence dashboard in -- that we did in collaboration together. But this time we wanted to know -- does building a strong foundation of data lead to good ARP investments?
So, as you know, in the ARP guidance, it encouraged state and local governments to use evidence of effectiveness. It defined evidence and it provided incentives for focusing on evidence-based programs. So answering this question -- does it matter if a local government has a good foundation of using data? Does it help them better invest federal dollars? So here's what we found. Certified cities were better positioned to use data and evidence to promote recovery, economic mobility, and racial equity through ARP plans. So that's really exciting. Specifically, cities with strong results-driven contracting practices developed investment plans that better adhered to the federal guidance for using data and evidence. And also certified cities, so the What Works Cities certification.
So those cities that had been certified, that effectively-engaged residents, strengthened the impact of federal dollars by prioritizing that engagement. So cities -- I'll just note that cities and states have until December of 2024 to obligate their ARP funds, and it's an incredible opportunity and responsibility to use these one-time funds to address racial inequalities and provide economic mobility outcomes for all residents. And we've really continued to see the power of data and evidence to make these changes and improve results.
[J.B. WOGAN]
As listeners may or may not know, you previously worked for Ben McAdams, who was a Congressman. Before that, I believe you were referencing him as the Mayor in South Lake County, Utah.
[NICHOLE DUNN]
Yeah.
[J.B. WOGAN]
And I remember -- he was a Moneyball for Government All-Star back in the day. Do you think -- I mean, you were talking earlier about that paper on the -- the impact of What Works Cities certification and how cities that have this foundation of evidence, or that being able to take advantage of federal money that has these evidence provisions. But I was wondering about another potential impact being that, as you've got these local officials who are gaining expertise in how to use evidence in their work, and not just expertise but that, you know, they're starting to appreciate the -- the -- the value of it, then they come to Washington. Do -- are you seeing more people like Ben McAdams who are -- have -- coming from -- from What Works Cities cities and then bringing that -- that background to the work that they do at the federal level?
[NICHOLE DUNN]
Yeah. That's a great question. So the way that I was connected to Results for America was that I was in the inaugural class of the local government fellowship when I was the Deputy Mayor and Chief Administrative Officer at Salt Lake County working for Mayor Ben McAdams then. And I was part of a Results for America fellowship. And that's really how we started to learn about the power of data and evidence, how we got really interested in doing things like "Pay for Success." And so I do think, once you -- we always talked about the Pay for Success Project. It was small in comparison to Salt Lake County's overall budget, but what was so powerful is it changed the way Ben McAdams thought about all of the funding at Salt Lake County.
And we started thinking about, okay, not just in early childhood or in, you know, substance abuse treatment -- what about in all areas of our funding? Are we getting a return on investment for taxpayer dollars? So, yeah, that definitely -- he carried that thinking into Congress and we -- we do see that. So, for example, Representative Greg Landsman worked with a great partner organization of ours, Strive Together, and really understood what it means to drive community-level outcomes. And he understood that evidence and focusing on data and tracking outcomes was how you -- you did that. And that is part of what drove his interest in being part of the -- the What Works caucus, as he sees it as, you know, a really powerful tool to make a difference for children and families in the communities he serves.
But a side note -- so many of the -- of our inaugural class of fellows are still doing this work. Oliver Wise is now at the Department of Commerce. Maia Jachimowicz, who was a fellow with me, is at RFA doing our equity work. You know, we have just -- I think once -- it's kind of, like, once you see it, it -- it's hard to think about it in a -- in a different way.
[J.B. WOGAN]
And I -- just because I happen to know those names, for listeners -- that would be, Oliver worked for Mayor Landrieu in New Orleans, and Maya worked for Mayor Nutter in Philadelphia.
[MIKE BURNS]
The analytical perspectives chapter of the budget also notes that progress and -- and building and driving evidence in the federal government requires widespread adoption of a culture of learning and experimentation. Knowing that agencies can be hesitant to experiment in favor of the status quo, what advice would you have for leaders to embrace this culture and help drive further gains in federal evidence building and use?
[NICHOLE DUNN]
Yeah. I think a couple of things. So one thing that Results for America has found working with federal agencies who are trying to foster a culture of learning is that agency leaders need to prioritize hiring evaluation staff and embedding them throughout their organization to avoid siloing evidence and data in -- you know, in specific areas. So as I mentioned before, we need to bring our evidence data and evaluation experts to the decision making table. And we need to bring everyone together to talk about learning.
You know, for example, the Substance Abuse and Mental Health Services Administration hosts monthly data parties to bring together staff and -- and talk about what they're learning and how it can be incorporated into their work. It's really crucial to ensure that evidence and data leaders within federal agencies have a say in policy decisions to integrate their work and -- and have those incentives for continuing to do this hard work. So we need to celebrate progress and -- and share stories of how evidence has made a difference in the programs we choose and how it makes a difference in the -- the impact that they have.
And then, again, I would just reiterate, you know, finding those places where we can really take what we've learned, take all of this great evidence we have and apply it. So, again, incorporating evidence provisions into federal grant making, using evidence in the -- the budget process. So I think I would emphasize this movement towards evidence use.
[MIKE BURNS]
And Congress certainly has an important pivotal role to play. Results for America was involved -- and I know we -- you and I have spoken at length about this -- but you were involved in the recent formation of the bipartisan Congressional What Works Caucus. Could you tell -- tell us a little bit more about the caucus and how they're approaching evidence and -- and data in the federal spending process?
[NICHOLE DUNN]
Yeah. We're really excited about this caucus. The members are super engaged and proactive. And they're really thinking about how to push evidence-based policy making forward. What's really exciting about this approach is that it's bipartisan by nature. It appeals to anyone who wants to improve government services, promote positive outcomes, and make sure taxpayer dollars are spent well. So, really, we're excited about the opportunity for this caucus to -- to shine. We're looking for -- we're working with them to identify where evidence-based policy making is happening in Congress, and lifting that up, looking for where it's not happening and where it should be happening.
And finding ways to -- to define evidence and -- and incentivize the use of evidence. Take the workforce space, for example. There's a major Worker Training Bill making its way through Congress, and everyone is onboard with the data and evidence provisions. Things like improving data quality, agreeing on what counts as evidence-based, and funding to help states' data systems. So this sounds wonky, as you said, but they're actually pretty appealing across party lines. So we're excited for finding more ways to -- you know, advance evidence-based policy in -- areas such as workforce, student achievement, and other areas.
And another thing that we're keeping a close eye on that I know you're aware of is this H.Con. Resolution 49. It's the congressional evidence-based policy making resolution. This is led by a bipartisan group of representatives, and this resolution aims to bring data and evidence into the lawmaking process. So this could mean legislation that really makes a difference in people's lives. So federal policies really rooted in evidence have a huge potential to make an impact, and really restore trust in government, and help, you know, people across the country see policies that work. So excited to keep working with them.
[MIKE BURNS]
And -- and what would you say, I mean, to someone -- someone skeptical. I mean, you see the -- the headlines, the -- you know, the bickering and the in -- the fighting in Congress which doesn't seem to square with this bipartisan push for evidence and data in decision making. What would you say to -- to someone who is skeptical, because on the -- they are so contradictory?
[NICHOLE DUNN]
Yeah. I would just say that there's a strong track record. We saw that evidence -- you know, we have seen bipartisan support for these type of efforts over the last ten years. We, you know -- at the federal event where we released the -- the Power of Evidence paper that we were discussing earlier, we had Senator Patty Murray and Former Speaker Paul Ryan talking about why they came together, you know, around this issue. It really just is an issue that transcends partisanship. I experienced this in my time in Salt Lake County government. We had, you know, a Democratic Mayor and a Republican Counsel.
And we -- this is an area we were able to make a lot of progress advancing -- an Early Childhood Pay for Success project and other innovative evidence-based programs around alternatives to incarceration. I think the -- you know what I would say to folks that are skeptical is, you know, look at the history. Look at the track record. It -- it has -- it has had bipartisan appeal and continues to work. And I -- I think it's just because it's one of those simple common sense things. If something works, it's pretty appealing.
[MIKE BURNS]
And it seems to be durable, too. I mean, you -- you mentioned Former Speaker Ryan. He's been out of office for a number of years now. Then you have the caucus recently standing up. You have folks like Congressman Derek Kilmer, you know, pushing these efforts forward. So what -- what's next? I mean, what's next for -- you think that these efforts in Congress or -- or for RFA, even?
[NICHOLE DUNN]
Yeah. I think that a couple of our big focuses is, one, continue to push federal grants to define and prioritize evidence. Continuing to shift dollars to what works. We are really focused on making progress in key economic mobility outcomes like third grade reading, postsecondary access and success, a living wage, and really we have seen that this can happen by ensuring that federal dollars, which the vast majority -- $1.2 trillion -- go out out in federal grants. So if we can make sure those grants focus the dollars on programs we know work, we believe we're going to see increases in advancements in those really important outcomes for people across the country.
So that is going to be an area we're very focused on is, you know, shifting dollars to what works to make -- really improve the impact in those outcome areas. And then we really will be focused in Congress on making sure new legislation or current legislation, like the WIOA reauthorization, has a definition of evidence that can help us figure out what works for who, in what circumstances, and then prioritizes dollars to focus on evidence. So we will be finding those opportunities and working with bipartisan members in Congress to advance those.
[MIKE BURNS]
Well, Nichole, I -- I want to thank you for your time, your partnership, your insights. We are certainly huge fans of RFA and of you, of course. Before I let you go, where can -- where can people find you and RFA?
[NICHOLE DUNN]
You can find us at results4america.org. And it's results, number 4, america-dot-org. So you can see our events and what's happening, and reach out to anyone on our team. So -- excited to hear from you.
[MIKE BURNS]
Thank you, Nichole.
[NICHOLE DUNN]
All right. Thank you.
[J.B. WOGAN]
Thanks again to our guest, Nichole Dunn. In the show notes, we include links to a range of online resources that Nichole referenced in the conversation. I also want to thank you for listening to another episode of On the Evidence, the Mathematica podcast. This episode was produced by the inimitable Rick Stoddard. If you're a fan of the show, please consider leaving us a rating and review wherever you listen to podcasts. To catch future episodes of the show, subscribe at mathematica.org/ontheevidence.
Show notes
Read an op-ed by Michele Jolin, the chief executive officer and co-founder of Results for America, about how bipartisan elected leaders and career civil servants across the country who have been quietly transforming how governments invest taxpayer dollars to achieve better results.
Read the research brief by Mathematica for Results for America, which shows how city governments with a history of prioritizing data-driven practices were more likely to use federal relief funds from the American Rescue Plan (ARP) to deepen their commitment to results-driven governance—with the goal of improving outcomes for residents through effective programs.
Watch a recording from the January 2024 virtual event hosted by Results for America to discuss the federal government’s progress in using evidence and data to get better results, which featured the release of a new report by Harvard University Professor Christina Ciocca Eller on federal evidence-based policymaking efforts.
Read the 2023 Invest in What Works State Standard of Excellence from Results for America, which showcases 194 efforts across 46 states to build and use evidence and data to improve residents’ lives.
Explore the ARP Data and Evidence Dashboard from Mathematica and Results for America, which highlights how state and local government leaders are investing one-time federal pandemic relief and economic recovery funds and using data and evidence to get better results.
