Shopping for Schools Is Hard

As school choice initiatives have increased options for students and their families, making effective choices has become more challenging. Even though the technology for presenting information is rapidly evolving, the amount of information and its complexity are growing even more quickly and can be overwhelming, even for sophisticated parents and students. For those with limited education or English language skills, difficulties digesting information about school choices can put the most vulnerable students at even more of a disadvantage.

Mathematica Is Helping

This project ran an experiment to learn how best to select and present school choice data so that it is easy to use and understand and leads to smart choices. In addition, policymakers in charge of centralized school lottery systems have no systematic guidance on how to provide usable, actionable information to families. Our aim was to conduct a carefully controlled experiment to generate the kind of evidence needed to provide this guidance.

Cutting-Edge Methods Can Identify Best Ways to Present Information

The project included the following:

• A design that allowed the study team to vary many factors simultaneously and systematically. The design used a Bayesian hierarchical model to analyze results from a full factorial experiment, which allowed the team to examine 72 different ways of displaying school choice information.
• Analysis of information or “choice architecture”—the way choices are presented, and the impact of presentation on decision making. Because information overload can be a problem, how information is structured and presented matters. Building from a review of existing school shopping websites, we examined how to structure information for consumers to produce the greatest levels of usability and comprehension, and to maximize the chance that parents select the best schools for their children.
• Development of practical guide for school districts. We produced a practical guide for districts and other entities (cities or counties, for example) that are responsible for presenting school choice information to parents and students. The guide will translate the experimental research findings into evidence-based principles for designing effective displays of school choice information.

Mathematica collaborated on this study with Jon Valant of the Brookings Institution and with Tembo, Inc., a design and analytics firm that produced the information displays that were tested. Funding for the experiment was provided by the U.S. Department of Education’s Institute of Education Sciences. Funding for the technical working papers and additional dissemination was provided by the Walton Family Foundation.

Results from this study are now available. Below is a list of resources:

“Nudging Parents to Choose Better Schools: The Importance of School Choice Architecture,” Mathematica Working Paper 65.

This working paper features findings of the school website design study and discusses policy implications.

“Presenting School Choice Information to Parents: An Evidence-Based Guide.”

This guide is for information providers who develop and distribute school directories, choice websites, and report cards. It details the impact that different design strategies can have on parents’ understanding, ease of use, and satisfaction with the information provided, and how it can affect parents’ choice of school.

“Presenting School Choice Information to Parents: An Evidence-Based Guide—Appendix.”

This technical appendix provides a full description of the study’s design, methods (including the analytic approach), and findings.

Other publications related to this study include:

“Shopping for Schools: Mapping Choice Architecture in the Education Marketplace,” Mathematica Working Paper 59.

“Beyond ‘Treatment versus Control’: How Bayesian Analysis Makes Factorial Experiments Feasible in Education Research,” Mathematica Working Paper 61.