Public Reporting of Quality Data

Evaluation of the Hospital Quality Alliance (HQA) and Premier Hospital Quality Incentive (HQI) Demonstration

Mathematica evaluated the HQA program, which involves public reporting of data on the quality of health care delivered by U.S. hospitals. The data are made available on a website, Hospital Compare, maintained by the Centers for Medicare & Medicaid Services (CMS). Evaluation components included:

• Discussions with a small sample of hospital CEOs, chief medical officers, and quality improvement directors on the impact of HQA on their quality improvement efforts and their decision to participate in the pilot phase of the HQA project
• Assessment of HQA program implementation
• Estimates of the proportion of Medicare beneficiaries who may obtain health-related information from the internet
• In-depth case studies of six hospitals undergoing change to identify ways to support hospital transformation
• Consumer testing of prospective new Hospital Compare measures
• Assistance in designing, organizing, and summarizing a Hospital Quality Summit that focused on the impact of public reporting on hospitals
• A media scan to gauge reaction to the launch of the Hospital Compare website
• A large national sample survey of hospital chief medical officers and quality improvement directors on the impacts of public reporting on hospital programs.

Read an issue brief from this study: "Hospital Compare Highlights Potential Challenges in Public Reporting for Hospitals"

Mathematica also conducted an early evaluation of the Premier HQID program, a demonstration focusing on financial incentives to improve the quality of hospital services. Nearly 300 hospitals in the Premier system volunteered for the program. CMS will pay a bonus to hospitals that fall into the top two deciles in the distribution of several quality indicators and may reduce Medicare payments to hospitals that fall into the lowest decile. Mathematica's evaluation included discussions with 34 hospital quality improvement directors, as well as officials at CMS, Premier, and CMS's data processing contractor. The evaluation assessed hospital recruitment, program implementation, and the impact of financial incentives on hospital quality improvement efforts. CMS continues to use our findings as it refines the design of its public reporting and hospital pay-for-performance programs.

Both studies were funded by CMS and conducted under contract to the Delmarva Foundation for Medical Care, Inc.

Evaluation of the Quality Performance Measurement Projects and Communications and Outreach Campaigns

Mathematica is evaluating the California HealthCare Foundation's (CHCF) efforts to increase the supply of, and bolster the demand for, usable measures of the quality of care in California. We are assessing (1) the development and implementation of quality measurement and public reporting systems for hospitals, nursing homes, Medicare health maintenance organizations, and organized medical groups; and (2) the outreach campaigns to community organizations, public- and private-sector policymakers, and the mass media to disseminate several of the resulting reports. We are also developing a map of current national quality measurement and reporting efforts in order to provide national context and source of comparison for CHCF's efforts. This process evaluation analysis is based on document reviews and extensive interviews with measurement experts, profiled providers, the providers' trade associations, community organizations, public and private policymakers, and national experts on quality of care. The project will produce two reports presenting the lessons learned about the economic, political, and technical considerations that affect the development and dissemination of information on quality of care.

Back to Top