Evaluation of Level 4 School Turnaround Efforts in Massachusetts - Part 2: Impact of School Redesign Grants

In April 2010, the Massachusetts Board of Elementary and Secondary Education adopted regulations to formalize the Massachusetts Department of Elementary and Secondary Education’s approach to engaging with all schools, and to the lowest-performing schools in particular, to improve student performance. Using a scale of 1 to 5, Level 4 represents the state's most struggling schools not under state control.

AIR conducted a mixed-methods evaluation of how Level 4 schools use School Redesign Grants (SRGs) and other supports to catalyze improvement, and how SRGs specifically affect student achievement. This report summarizes findings from our impact analysis of how SRG receipt affects student achievement. (A separate report  describes how Level 4 schools implement key turnaround practices, and which specific strategies characterize schools able to improve student outcomes.)

The results suggest that the disbursement of federal Title I School Improvement Grants in the process designed by ESE (as SRGs) have consistently positive effects on student academic achievement. Moreover, these results are generally robust across districts, grade levels, and cohorts of grant recipients, and they are particularly strong for students who are ELLs or have free or reduced-price lunch status.
 

Key Findings

The impact study found the following:

• Students in the SRG schools performed better on the English Language Arts (ELA) and mathematics sections of the Massachusetts statewide student assessment compared with students in comparison schools.
• Positive effects of SRG on achievement were found for elementary school students in Grades 3–5 and middle school students in Grades 6–8 across all three years after implementation on both the ELA and mathematics sections. For high school students in Grade 10, positive effects were found across all three years for mathematics, but not for ELA.
• Positive effects on both the ELA and mathematics sections were found in all three years of program implementation in Boston and Springfield as well as for all remaining districts combined.
• SRG receipt was associated with a decrease in the achievement gap on both the ELA and mathematics sections between ELL and non-ELL students and between students who did and did not have FRPL status in SRG schools, as compared with the change in the achievement gap between students in the comparison schools.
• Subgroup analysis by cohort also found positive and statistically significant impacts one, two, and three years later for all cohorts contributing to the analysis in both ELA and mathematics.