|
Education: Latest Work
Reports | Journal Articles | Issue Briefs | Other
Reports |
|
|
| “High School Reform in Boston Public Schools: The Effect of Focus on High Schools on Student Academic Outcomes.” Susanne James-Burdumy, Irma Perez-Johnson, and Sonya Vartivarian, June 2008. The Focus on High Schools initiative is intended to transform Boston’s 12 comprehensive high schools into more effective institutions by breaking them down into smaller, more intimate learning communities and reforming their English and language arts programs. This report focuses on changes in students’ academic outcomes after implementation of the initiative. The study found reductions in math and English scores. On the positive side, days absent and tardy were lower, along with student suspensions. There was also an overall increase in promotions to the next grade, concentrated among special education students. |
|
“Passport to Teaching: Career Choices and Experiences of American Board Certified Teachers.” Steven Glazerman, Elizabeth Seif, and Gail Baxter, June 2008. A growing number of states now recognize an alternative teaching credential called the Passport to Teaching, offered by the American Board for Certification of Teacher Excellence (ABCTE). This new report describes the experiences of those who obtain the credential. Mathematica’s survey of more than 500 Passport holders reveals that nearly 6 out of 10 were teaching K-12 in the U.S.; of this group, 71 percent were in public schools, 12 percent in charter schools, and 15 percent in private schools. Passport teachers remain in the classroom at comparable rates to other teachers, with 85 percent of teachers who were certified by ABCTE through 2005 still in the classroom during the 2007-2008 school year.
|
|
| "Guidelines for Multiple Testing in Impact Evaluations of Educational Interventions." Peter Z. Schochet, May 2008. Statistical procedures that correct for multiple testing typically result in hypothesis tests with reduced statistical power because adjustment methods reduce the likelihood of identifying real differences between contrasted groups. There is disagreement among researchers about the use of multiple testing procedures and the appropriate trade-off between type I error and statistical power (type II error). These guidelines were developed to handle multiple testing in education research. In addition, the report provides details on the nature of the multiple testing problem and the statistical solutions that have been proposed; the creation of composite outcomes measures; and the Bayesian hypothesis testing approach. |
|
| "The Nation's Report Card: Technical Report of the NAEP Mathematics Assessment in Puerto Rico: Focus on Statistical Issues." G.P. Baxter, S. Ahmed, E. Sikali, T. Waits, M. Sloan, and S. Salvucci, September 2007. In 2003, a trial National Assessment of Educational Progress mathematics assessment was administered in Spanish to public school students in grades 4 and 8 in Puerto Rico. On the basis of preliminary analyses of the 2003 data, changes were made in administration and translation procedures for 2005. This report describes the content and administration of the trial assessments in Puerto Rico in 2003 and 2005, problems with item misfit in the 2003 data, results of a special validity analysis, and plans to integrate Puerto Rico into the national sample in future administrations. |
|
| "Evaluation of the Teacher Advancement Program (TAP) in the Chicago Public Schools: Study Design Report." Steven Glazerman, Allison McKie, Nancy Carey, and Dominic Harris, November 2007. Recent evidence has confirmed that teacher quality is a critical component in student achievement. The Teacher Advancement Program (TAP) is a whole-school approach to evaluating and compensating teachers and providing professional development opportunities to both improve teaching and help schools attract and retain good teachers. This report describes Mathematica’s five-year evaluation, which began in 2007, in high-need Chicago public schools. |
|
| “Measuring School Effectiveness in Memphis.” Kevin Booker and Eric Isenberg, April 2008. New Leaders for New Schools, a nonprofit organization committed to training school principals, is heading an initiative that offers financial awards to effective educators. This report details the value-added model Mathematica developed to identify effective schools in the Memphis City school district during the first year of the project. The model estimates each school’s effect on student performance across all tested grades and subjects, accounting for student mobility, observable differences in student demographics, and measurement error in test scores. |
|
| "Achievement and Attainment in Chicago Charter Schools." Kevin Booker, Brian Gill, Ron Zimmer, and Tim R. Sass, May 2008. During the past decade, charter schools have been one of the fastest growing segments of the K–12 education sector. This is the first report to examine how charter schools may affect long-term attainment, including high-school graduation and college entry. The authors find that Chicago's multi-grade charter high schools (those that include grades 6-12, 7-12, or K-12) appear to produce substantial positive effects on ACT scores, probability of graduating, and probability of enrolling in college. The large positive results suggest promise for multi-grade charter high schools (and perhaps charter high schools more generally) and demonstrate that evaluations limited to test scores may fail to capture important benefits of charter schools. |
|
| "Title I School Choice and Supplemental Educational Services: Interim Report." Brian Gill, Jennifer Sloan McCombs, Scott Naftel, Karen Ross, Mengli Song, Jennifer Harmon, and Georges Vernez. State and Local Implementation of the No Child Left Behind Act: Volume IV, 2008. This report presents findings on the implementation of parental choice options from the first year of two national studies of No Child Left Behind. In 2004-2005, nearly 6.2 million students were eligible for Title I school choice and as many as 1.8 million were eligible for supplemental educational services. Only about one percent took advantage of the school choice option, and about 17 percent took advantage of supplemental services. In a subsample of large urban districts, the average achievement of the schools chosen by students using the school choice option was consistently higher than the average achievement of the schools they came from, and their parents were largely satisfied with the new schools. The report also notes that low participation rates in Title I school choice and supplemental educational services may be related to problems communicating with parents. |
|
|
"National Assessment of Title I Final Report. Volume II, Closing the Reading Gap: Findings from a Randomized Trial of Four Reading Interventions for Striving Readers." Joseph Torgesen, Allen Schirm, Laura Castner, Sonya Vartivarian, Wendy Mansfield, David Myers, Fran Stancavage, Donna Durno, Rosanne Javorsky, and Cinthia Haan, October 2007. This report details findings from an experimental evaluation of four widely used programs for elementary school students with reading problems—Corrective Reading, Failure Free Reading, Spell Read P.A.T., and Wilson Reading. Conducted just outside Pittsburgh, Pennsylvania, in the Allegheny Intermediate Unit, the evaluation explored the extent to which the programs affect both word-level reading and reading comprehension skills of students in grades three and five identified as struggling readers. This report presents impacts on reading test scores one year after the interventions ended, when most of the students were in fourth and sixth grade, as well as impacts on state reading and mathematics test scores from the previous year. Key findings show that the interventions improved some reading skills but did not improve state test scores. Furthermore, younger students benefited more, and the interventions narrowed some reading gaps. Appendices |
|
|
"The Postsecondary Achievement of Participants in Dual Enrollment: An Analysis of Student Outcomes in Two States.” Melinda Mechur Karp, Juan Carlos Calcagno, Katherine L. Hughes, Dong Wook Jeong, and Thomas R. Bailey, October 2007. Dual enrollment programs enable high school students to enroll in college courses and earn college credit. Once limited to high-achieving students, such programs are increasingly seen as a means to support the postsecondary preparation of average-achieving students. Moreover, though dual enrollment programs typically have been reserved for academically focused students, increasing numbers of career and technical education (CTE) programs are providing such opportunities to their students. In this report, the authors examine the impact of dual enrollment participation for students in Florida and New York City, examining postsecondary outcomes for participating CTE students in both locations and outcomes of participation for all students in Florida. The report notes that dual enrollment is a useful strategy for encouraging postsecondary success for all students, including those in CTE programs. |
|
| Rhetoric vs. Reality: What We Know and What We Need to Know About Vouchers and Charter Schools. Brian Gill, P. Mike Timpane, Karen Ross, Dominic Brewer, and Kevin Booker, August 2007. Vouchers and charters are two of the most ambitious attempts to improve school performance through parental choice and privatization of governance. The updated edition of this book (which was first released in 2001) summarizes new empirical evidence on the achievement impacts of voucher programs and charter schools operating in various locations across the country (and, to some extent, around the world). It concludes that neither the hopes of the supporters nor the fears of the skeptics have been realized, but that well-designed voucher and charter programs have shown promising signs, while few have shown any evidence of substantial harms to student achievement. The authors note that the details of policy design are likely to be critical to the effects produced by choice programs: not all voucher programs and charter laws are alike, and different programs will produce different effects. They call for additional research to help clarify the importance of specific policy elements and to assess a deeper and wider range of student outcomes. |
|
Journal Articles |
|
|
| “Statistical Power for Random Assignment Evaluations of Education Programs.” Peter Z. Schochet, Journal of Educational and Behavioral Statistics, March 2008. This article examines theoretical and empirical issues related to the statistical power of impact estimates for experimental evaluations of education programs. The author considers designs where random assignment is conducted at the school, classroom, or student level, and employs a unified analytic framework using statistical methods from the literature. Focusing on standardized test scores of elementary school students, this article discusses appropriate precision standards and, for each design, the required number of schools to achieve those standards using empirical values of intraclass correlations, regression R2 values, and other parameters. Clustering effects vary by design but are typically large. As a result, large school samples are required for education trials, and many evaluations will have sufficient power to detect precise impacts only for relatively large subgroups of sites. |
|
| “After-School Program Effects on Behavior: Results from the 21st Century Community Learning Centers Program National Evaluation.” Susanne James-Burdumy, Mark Dynarski, and John Deke, Economic Inquiry, January 2008. This paper presents evidence on after-school programs’ effects on behavior from the national evaluation of the U.S. Department of Education’s 21st Century Community Learning Centers after-school program. Findings come from both of the study’s components: (1) an elementary-school component based on random assignment of 2,308 students in 12 school districts, and (2) a middle-school component based on a matched comparison design including 4,264 students in 32 districts. Key findings include higher levels of negative behavior for elementary students and some evidence of higher levels of negative behaviors for middle school students. |
|
| "Academic Achievement and School Functioning Among Non-Incarcerated Youth Involved with the Juvenile Justice System." Jonathan D. Brown, Anne W. Riley, Christine M. Walrath, Philip J. Leaf, and Carmen Valdez, Journal of Education for Students Placed at Risk, January-March 2008. This article reports on the education problems of youth involved with the juvenile justice system but not incarcerated. More than half demonstrated deficits in academic functioning, with standard achievement scores as low as five standard deviations below the normative mean. Non-Caucasian youth and those who received special education services or lived in an urban area had lower achievement. These findings suggest that youth involved with the justice system but not incarcerated demonstrate problems in academic achievement similar to incarcerated youth and may benefit from targeted education interventions. |
|
|
"Governing the Family Through Education: A Genealogy on the Home/School Relation." Kirsten Kainz and Nikki L. Aikens, Equity & Excellence in Education, October 2007. The authors examine a century of rhetoric on the home/school relation and associated parent involvement. The review investigates how current ideas about this relationship and parent involvement might inhibit the realization of justice, equity, and excellence in education. |
|
| "When Schools Stay Open Late: Results from the National Evaluation of the 21st Century Community Learning Centers Program." Susanne James-Burdumy, Mark Dynarski, and John Deke, Educational Evaluation and Policy Analysis, December 2007. This article presents evidence from the elementary school component of Mathematica’s national evaluation of the 21st Century Community Learning Center afterschool programs. The findings indicate that the programs affected the supervision students received after school, with parents less likely to care for their child and other adults more likely, but there was no statistically significant effect on the incidence of self-care. Students in the program reported feeling safer after school, but their academic outcomes were not affected, and they had more incidents of negative behavior. The elementary study was conducted in 12 school districts and 26 afterschool centers. |
|
|
"Stepping Stones to a Degree: The Impact of Enrollment Pathways and Milestones on Community College Student Outcomes." Juan Carlos Calcagno, Peter Crosta, Thomas Bailey, and Davis Jenkins, Research in Higher Education, November 2007. This paper presents findings from a study of the experiences and outcomes of older and younger community college students. The authors developed a discrete-time hazard model using longitudinal transcript data on a cohort of first-time community college students in Florida to compare the impact of enrollment pathways (such as remediation) and enrollment milestones (such as attaining a certain number of credits) on educational outcomes of older students—those who enter college for the first time at age 25 or later—with those of traditional-age students. Results suggest that reaching milestones such as obtaining 20 credits or completing 50 percent of a program is a more important positive factor affecting graduation probabilities for younger students than it is for older students. |
|
Issue Briefs |
|
"Expanding Beyond Academics: Who Benefits and How?" John Deke and Joshua Haimson, September 2006. The growing use of math and reading scores to measure school and student performance, spurred by the No Child Left Behind Act, has heightened an old debate about which competencies public schools should encourage students to develop. This issue brief looks at how students' competencies in high school relate to postsecondary educational attainment and earnings later in life. These competencies include academic achievement (as measured by test scores), leadership skills, sports-related skills, work habits, prosocial behavior, and locus of control (a measure of students' belief that they control their future). The brief concludes that many of these competencies appear to predict students' later success in higher education and the labor market. It also concludes that not all students would benefit from improving the same competencies, suggesting that an individualized approach to education may be preferable to a "one-size-fits-all" approach.
Full Report |
|
Other |
|
| “Going Beyond Test Scores: Evaluating Charter School Impact on Educational Attainment in Chicago and Florida.” Kevin Booker, Tim R. Sass, Brian Gill, and Ron Zimmer, August 2008. Unlike past charter school studies focused on student achievement, the authors analyzed the relationship between attendance and educational attainment in charter high schools. They found that these schools in Florida and Chicago had substantial positive effects on both high school completion and college attendance. For students who attended a charter middle school, those who went on to attend a charter high school were 7 to 15 percentage points more likely to earn a standard diploma than students who attended a traditional public high school. Similarly, those attending a charter high school were 8 to 10 percentage points more likely to attend college. |
|
Publications from the What Works Clearinghouse (WWC), which is administered by Mathematica for the Institute of Education Sciences:
Intervention Reports
Houghton Mifflin Reading (September 2008)—Beginning Reading report designed to be used as a full-year curriculum program, the Houghton Mifflin Reading© system aims to develop oral language and comprehension, phonemic awareness, decoding skills (phonics, analogy, context, and word recognition), fluency, reading comprehension, writing, spelling, and grammar.
Breakthrough to Literacy (September 2008)—Is a curriculum for students in preschool through third grade that introduces them to a book-a-week throughout the year. The book serves as a focal point for classroom activities with whole group and small group instruction. The curriculum also offers independent learning activities, including computer-based instruction, that allow students to progress at their own pace.
Mathematics in Context (September 2008)—Is a middle school mathematics curriculum for grades 5 through 8. The curriculum teaches students to explore the relationships among different domains of mathematics (such as algebra and geometry) and to develop strategies for reasoning through problems, encouraging students to collaborate on problem solving.
The New Century High Schools Initiative (September 2008)—Is a program designed to improve large, underperforming high schools by transforming them into small schools with links to community organizations. The schools choose a curriculum that has a theme or career focus, such as engineering, health science, or theater. Each school partners with a community organization that can participate in curriculum development, school management, after-school activities, or other operational aspects of the school.
Reading Mastery (August 2008)—A full-year curriculum designed to provide explicit, systematic instruction in English language reading. The program teaches phonemic awareness, sound-letter correspondence, word and passage reading, vocabulary development, comprehension, and oral reading fluency.
Open Court Reading (August 2008)—An elementary basal reading program for grades K-6. The program is designed to follow a logical progression, systematically and explicitly teaching decoding, comprehension, inquiry, investigation, and writing.
Accelerated Middle Schools (July 2008)—Self-contained academic programs to help middle school students who are one to two years behind grade level catch up to their peers, give students the opportunity to cover an additional year of curriculum during their one to two years in the program. Programs can be structured as separate schools or as schools within a traditional middle school. The WWC found accelerated middle schools to have potentially positive effects on staying in school and positive effects on progressing in school.
Practice Guides
Dropout Prevention (September 2008)—It provides six recommendations for reducing dropout rates in high schools and middle schools. Designed for educators, administrators, and policymakers, the guide offers strategies for diagnosing dropout problems, implementing intervention programs to improve behavior and social skills, and keeping students engaged.
Improving Adolescent Literacy: Effective Classroom and Intervention Practices (September 2008)—This practice guide presents practical strategies that classroom teachers and specialists can use to help students in grades 4 to 12 refine and build their reading skills. The recommendations can be integrated into classroom instruction to help students gain more from their reading tasks, improve motivation for and engagement in learning, and assist struggling readers who may need intensive and individualized attention.
Turning Around Chronically Low-Performing Schools (May 2008)—The guide identifies practices that can improve the performance of chronically low-performing schools—a process commonly referred to as creating "turnaround schools." The four recommendations in the guide aim to help failing schools make adequate yearly progress.
Quick Reviews
Promoting Broad and Stable Improvements in Low-Income Children’s Numerical Knowledge Through Playing Number Board Games (August 2008)—Reviewed a study of whether playing number board games improved numeracy skills of low-income preschoolers.
The Effect of Performance-Pay in Little Rock, Arkansas on Student Achievement (August 2008)—Examined a study of the Achievement Challenge Pilot Project, a performance-pay program for teachers to improve elementary students’ academic achievement.
Paying for A’s: An Early Exploration of Student Reward and Incentive Programs in Charter Schools (August 2008)—Examined a study of whether offering student reward and incentive programs in charter schools affects academic achievement.
Charter School Performance in Los Angeles Unified School District: A District and Neighborhood Matched Comparison Analysis (August 2008)—Looked at a study of whether Los Angeles charter schools have higher growth in student achievement than traditional public schools.
Making a Difference? The Effects of Teach For America in High School (July 2008)—This study looked at whether having a Teach For America (TFA) teacher instead of a non-TFA teacher affects the academic performance of high school students.
The Advantage of Abstract Examples in Learning Math (July 2008)—This study examined whether college students are better able to apply knowledge of simple mathematical concepts when they are taught the concepts using abstract symbols or concrete examples.
A Little Now for a Lot Later: A Look at a Texas Advanced Placement Incentive Program (June 2008)—Examined whether the program affects students' SAT/ACT scores and college attendance.
The Negative Impacts of Starting Middle School in Sixth Grade (June 2008)—Considered whether placing sixth graders in middle schools rather than elementary schools affects their behavior or academic performance.
Scaling Up SimCalc Project: Can a Technology Enhanced Curriculum Improve Student Learning of Important Mathematics? (May 2008)—Assessed whether SimCalc Mathworlds™ improves students' knowledge of the algebra concepts of rate and proportionality.
Outcomes of a Prospective Trial of Student-Athlete Drug Testing: The Student Athlete Testing Using Random Notification (SATURN) Study (May 2008)—Examined whether the SATURN program affects illicit drug and alcohol use among student athletes.
Outcomes Linked to High-Quality Afterschool Programs: Longitudinal Findings from the Study of Promising Afterschool Programs (May 2008)—Assessed whether high-quality after-school programs operating in high-poverty communities improve the academic, social, and behavioral outcomes of participating students.
To find all WWC-related publications, please go to the WWC Publications page.
|
|
| “Evaluating the Performance of Philadelphia’s Charter Schools.” Ron Zimmer, Suzanne Blanc, Brian Gill, and Jolley Christman, March 2008. Charter schools are publicly funded schools that operate autonomously, outside the direct control of local school districts, and generally enroll students through the choices of their parents rather than through residential assignment. These schools have proved popular nationwide. The authors examine effects of charter schools on reading and mathematics achievement for students in Philadelphia. They also examine effects of years of operation, grades served, mission, and demographics on student achievement; types of students charter schools attract; turnover rates; and charter schools’ impact on student achievement in traditional public schools. Achievement gains of students attending charter schools are approximately equal to the gains of students attending district-operated public schools. There is little evidence that charter schools either help or harm the achievement of students in nearby district schools. |
|
| “Missouri’s Teacher Career Ladder Program.” Kevin Booker and Steven Glazerman, February 2008. Although Missouri has had a Career Ladder program for teachers since 1987, there has been little research examining the program’s effects. This working paper, presented at a National Center on Performance Incentives conference, examines effects on student achievement, using longitudinal data on district math and reading scores for 524 Missouri school districts over a nine-year period. The paper compares achievement levels in participating districts with those of a matched group of nonparticipating districts. The estimated effects range from small positive effects to no effect. The authors conclude that, if the Career Ladder has a positive effect on test scores, it is probably very small. A soon-to-be-completed companion paper will explore program operations, and another will examine the relationship between Career Ladder participation and teacher retention. |
|
| "School Competition and Student Outcomes." Brian Gill and Kevin Booker. In Handbook of Research in Education Finance and Policy, 2008. A variety of policies can promote market forces in education, potentially producing competitive effects. These effects—positive or negative—are among the most important outcomes produced by educational privatization and school choice. This chapter addresses issues related to the effects of competition on conventional public schools and the traditional public purposes of education, including student integration and education of citizens.
To find all education-related publications, please go to the Publications Search page.
|
|
Back to Top |
