Developing Administrative Databases for Program Evaluation, Monitoring, and Design

Mathematica is a leader in helping public agencies use their administrative databases to evaluate, monitor, and design programs for vulnerable populations. Although many agencies collect extensive information for operational purposes about the people they serve, they can find it difficult to use these data for research. We address this problem by organizing, cleaning, and enhancing administrative databases so that they can be used to inform policy debates on whether programs are achieving their goals or need improvement. This approach to program evaluation can be much more cost-effective than collecting original survey data. The following examples illustrate the scope of our work.

Developing Administrative Databases

Making Databases Useful for Program and Policy Evaluation. We work with all 50 state Medicaid agencies to correct missing and inconsistent data in their Medicaid State Information System (MSIS) files. We also help the Centers for Medicare & Medicaid Services (CMS) reorganize and reformat MSIS data into the Medicaid Analytic eXtract (MAX) files for program and policy analysis. CMS is also using MAX data to assess the effectiveness of a new program designed to enhance employment options for people with disabilities.

Linking Databases to Make Them More Powerful. Our staff has linked more than 100 extracts from Social Security Administration administrative records to develop a longitudinal research database that includes the more than 17 million working-age people who have received Social Security Disability Insurance or Supplemental Security Income benefits. We have also combined data from more than 30 states with Medicaid Buy-In programs to develop a national database about participation in this new program.

Protecting Privacy. Even if an individual's name and address are removed from an administrative database, it may still contain enough information to identify the individual and compromise his or her privacy. We have developed approaches for determining how much data on individuals must be deleted or otherwise masked in a database to protect individual privacy.

Using Administrative Databases

Evaluating Programs and Policies. We have used linked data from Social Security Administration records to estimate the impacts of the Ticket to Work program on the employment and benefit receipt of people who receive disability benefits.

Generating Richer Population Descriptions. By linking data from several agencies, our researchers have helped agencies get a far richer understanding of the people they serve than is possible from a single agency's data. For example, we linked earnings data maintained by the Social Security Administration with data from states and from Medicare and Medicaid claims/eligibility files to help CMS clarify whether those enrolled in the Medicaid Buy-In program are working as intended by the legislation.

Monitoring Program Performance. In 1997, the U.S. Department of Labor (DOL) outsourced data processing required by the Employee Retirement and Income Security Act (ERISA) to a contractor. Since then, we have used administrative data on employer-sponsored employee benefits (primarily pensions and health insurance) to monitor the vendor's performance, and DOL has used our reports to improve the program.

Simulating and Projecting Programs and Policies. We are combining state administrative and survey data to estimate the costs and effectiveness of different programs to reduce the number of families without health insurance. We developed a computer model to simulate what would happen under different scenarios, and several states have used information from the model to develop new health insurance programs.

Designing Programs and Policies. We are using Medicare claims, enrollment, and quality data to assist CMS in designing the Physician Quality Reporting Initiative (PQRI). PQRI provides physicians with a financial incentive to report data on the quality of the health care they provide. CMS used our results to design a data validation system for PQRI.

Understanding How Precise the Numbers Are

Estimating Precision. Although knowing the precision of an estimate is essential to understanding whether to base policy decisions on it, estimating precision is notoriously difficult. We use numeric and analytic approaches to estimate confidence regions around estimates that are based on administrative data.

Estimating Numbers for Small Areas or Small Groups. We combine Food Stamp Program administrative data with survey data to estimate the number of eligible families in small geographic areas across the country. The U.S. Department of Agriculture uses these estimates to allocate billions of dollars of funding to states and local areas.

Back to Top