Lessons Learned from Evaluation of Locally Driven Innovations in Health Care Delivery

The programs varied widely in the interventions provided, populations served, types of organizations involved, and number of beneficiaries enrolled. The diversity of the service delivery models provided a unique opportunity to test a wide range of locally driven innovations simultaneously, as well as determine whether any shared program or awardee characteristics were associated with a greater likelihood of reducing the need for costly health care services across a range of settings and populations. 

Mathematica and its partners conducted a comprehensive implementation and impact evaluation of the 38 awardees that remained in the program after the first year. We used a mixed-methods study design, tailored to the intervention features of each program, to assess whether the innovations reduced hospitalizations and emergency department (ED) visits, improved care quality, and lowered the total cost of care. We analyzed Medicare and Medicaid claims data; conducted surveys with program staff, physicians, and beneficiaries; conducted qualitative interviews with program administrators and frontline service providers; and analyzed program data collected for CMS.

Only four of the 19 programs for which we could produce valid impact estimates demonstrated statistically significant effects on hospital inpatient and ED service use and expenditures. When favorable results were achieved, they tended to be realized more through a reduction in emergency room visits than in hospitalizations and concentrated among higher-risk Medicare and Medicaid beneficiaries most likely to benefit from enhanced intervention services.

Service delivery models targeting patients with substantial nonmedical needs, using nonclinical staff, and focusing on individual patient care rather than transforming provider practice were more likely than programs without these features to reduce acute care and ED service use and lower medical spending. Health care interventions that included behavioral health, telehealth, and health information technology components were also more likely than programs without these components to reduce the need for costly health care services.