Education Policy Research

"Guidelines for Multiple Testing in Impact Evaluations of Educational Interventions." Peter Z. Schochet, May 2008. Statistical procedures that correct for multiple testing typically result in hypothesis tests with reduced statistical power because adjustment methods reduce the likelihood of identifying real differences between contrasted groups. There is disagreement among researchers about the use of multiple testing procedures and the appropriate trade-off between type I error and statistical power (type II error). These guidelines were developed to handle multiple testing in education research. In addition, the report provides details on the nature of the multiple testing problem and the statistical solutions that have been proposed; the creation of composite outcomes measures; and the Bayesian hypothesis testing approach.

"The Nation's Report Card: Technical Report of the NAEP Mathematics Assessment in Puerto Rico: Focus on Statistical Issues." G.P. Baxter, S. Ahmed, E. Sikali, T. Waits, M. Sloan, and S. Salvucci, September 2007. In 2003, a trial National Assessment of Educational Progress mathematics assessment was administered in Spanish to public school students in grades 4 and 8 in Puerto Rico. On the basis of preliminary analyses of the 2003 data, changes were made in administration and translation procedures for 2005. This report describes the content and administration of the trial assessments in Puerto Rico in 2003 and 2005, problems with item misfit in the 2003 data, results of a special validity analysis, and plans to integrate Puerto Rico into the national sample in future administrations.

"Evaluation of the Teacher Advancement Program (TAP) in the Chicago Public Schools: Study Design Report." Steven Glazerman, Allison McKie, Nancy Carey, and Dominic Harris, November 2007. Recent evidence has confirmed that teacher quality is a critical component in student achievement. The Teacher Advancement Program (TAP) is a whole-school approach to evaluating and compensating teachers and providing professional development opportunities to both improve teaching and help schools attract and retain good teachers. This report describes Mathematica’s five-year evaluation, which began in 2007, in high-need Chicago public schools.

“Measuring School Effectiveness in Memphis.” Kevin Booker and Eric Isenberg, April 2008. New Leaders for New Schools, a nonprofit organization committed to training school principals, is heading an initiative that offers financial awards to effective educators. This report details the value-added model Mathematica developed to identify effective schools in the Memphis City school district during the first year of the project. The model estimates each school’s effect on student performance across all tested grades and subjects, accounting for student mobility, observable differences in student demographics, and measurement error in test scores.

"Achievement and Attainment in Chicago Charter Schools." Kevin Booker, Brian Gill, Ron Zimmer, and Tim R. Sass, May 2008. During the past decade, charter schools have been one of the fastest growing segments of the K–12 education sector. This is the first report to examine how charter schools may affect long-term attainment, including high-school graduation and college entry. The authors find that Chicago's multi-grade charter high schools (those that include grades 6-12, 7-12, or K-12) appear to produce substantial positive effects on ACT scores, probability of graduating, and probability of enrolling in college. The large positive results suggest promise for multi-grade charter high schools (and perhaps charter high schools more generally) and demonstrate that evaluations limited to test scores may fail to capture important benefits of charter schools.

"Title I School Choice and Supplemental Educational Services: Interim Report." Brian Gill, Jennifer Sloan McCombs, Scott Naftel, Karen Ross, Mengli Song, Jennifer Harmon, and Georges Vernez. State and Local Implementation of the No Child Left Behind Act: Volume IV, 2008. This report presents findings on the implementation of parental choice options from the first year of two national studies of No Child Left Behind. In 2004-2005, nearly 6.2 million students were eligible for Title I school choice and as many as 1.8 million were eligible for supplemental educational services. Only about one percent took advantage of the school choice option, and about 17 percent took advantage of supplemental services. In a subsample of large urban districts, the average achievement of the schools chosen by students using the school choice option was consistently higher than the average achievement of the schools they came from, and their parents were largely satisfied with the new schools. The report also notes that low participation rates in Title I school choice and supplemental educational services may be related to problems communicating with parents.

More Reports

"Expanding Beyond Academics: Who Benefits and How?" John Deke and Joshua Haimson, September 2006. The growing use of math and reading scores to measure school and student performance, spurred by the No Child Left Behind Act, has heightened an old debate about which competencies public schools should encourage students to develop. This issue brief looks at how students' competencies in high school relate to postsecondary educational attainment and earnings later in life. These competencies include academic achievement (as measured by test scores), leadership skills, sports-related skills, work habits, prosocial behavior, and locus of control (a measure of students' belief that they control their future). The brief concludes that many of these competencies appear to predict students' later success in higher education and the labor market. It also concludes that not all students would benefit from improving the same competencies, suggesting that an individualized approach to education may be preferable to a "one-size-fits-all" approach. Full Report


Back to Top