A Method for Measuring the Effect of Certified Electronic Health Record Technology on Childhood Immunization Status Scores Among Medicaid Managed Care Network Providers

Publisher: Journal of Biomedical Informatics, vol. 110
Oct 01, 2020
Paul Messino, Hadi Kharrazi, Julia M. Kim, and Harold Lehmann
  • Reusable method for estimating impact of CEHRT on health care quality.
  • Method applies Bayesian analysis, estimates impact of CEHRT on immunization scores.
  • No statistically significant difference in the odds of meeting immunization score.


To provide a methodology for estimating the effect of U.S.-based Certified Electronic Health Records Technology (CEHRT) implemented by primary care physicians (PCPs) on a Healthcare Effectiveness Data and Information Set (HEDIS) measure for childhood immunization delivery.


This study integrates multiple health care administrative data sources from 2010 through 2014, analyzed through an interrupted time series design and a hierarchical Bayesian model. We compared managed care physicians using CEHRT to propensity-score matched comparisons from network physicians who did not adopt CEHRT. Inclusion criteria for physicians using CEHRT included attesting to the Childhood Immunization Status clinical quality measure in addition to meeting “Meaningful Use” (MU) during calendar year 2013. We used a first-presence patient attribution approach to develop provider-specific immunization scores.


We evaluated 147 providers using CEHRT, with 147 propensity-score matched providers selected from a pool of 1253 PCPs practicing in Maryland. The estimate for change in odds of increasing immunization rates due to CEHRT was 1.2 (95% credible set, 0.88–1.73).


We created a method for estimating immunization quality scores using Bayesian modeling. Our approach required linking separate administrative data sets, constructing a propensity-score matched cohort, and using first-presence, claims-based childhood visit information for patient attribution. In the absence of integrated data sets and precise and accurate patient attribution, this is a reusable method for researchers and health system administrators to estimate the impact of health information technology on individual, provider-level, process-based, though outcomes-focused, quality measures.


This research has provided evidence for using Bayesian analysis of propensity-score matched provider populations to estimate the impact of CEHRT on outcomes-based quality measures such as childhood immunization delivery.

Senior Staff

Paul Messino
Read More