September 21, 2012 MEMORANDUM To: Joseph Belluck, SUNY Trustee, Chair, Charter Schools Committee Stephen Hunt, SUNY Trustee, Member, Charter Schools Committee John Murad, SUNY Trustee, Member, Charter Schools Committee From: Susan Miller Barker, Executive Director Subject: Academic Performance of SUNY Authorized Charter Schools Students in charter schools authorized by the SUNY Board of Trustees post strong performance on the state’s English language arts and mathematics exams. After the 2008-09 school year, the State Education Department adjusted the proficiency cut scores and the timing of when schools administer the exams during the school year. These adjustments notwithstanding, SUNY charter schools outperform their districts of location and schools chartered by other authorizers. The Charter Schools Institute examines two important comparative measures to determine the overall performance of its schools. The Institute calculates each school’s performance relative to the performance of all schools within its district of location. The Institute also calculates each school’s Effect Sizes, which measure the effectiveness of each school’s instructional program in English language arts and mathematics while controlling for the percentage of students in the school who are eligible for the federal Free Lunch Program. The Institute also uses these measures to examine the performance of individual schools particularly in the context of making recommendations to the SUNY Board of Trustees about renewal decisions. High schools and schools serving only grades K-2 are excluded from these analyses because New York State administers its testing program to students in grades 3 through 8. The following charts highlight the performance of SUNY authorized charter schools on the state’s 2011-12 state assessments. Figure 1: Difference between SUNY Authorized Charter School and Local District Percents Proficient in ELA; Grades 3-8; 2012 83 percent of SUNY’s charter schools outperform their district of location in ELA. 37.5 percent of Figure 2 shows the difference between the performance of SUNY’s charter schools on the state’s mathematics exam and the performance of their local school districts on the same exam in 2011-12. The schools highlighted in blue are under consideration for charter renewal in 2012. Figure 2: Difference between SUNY Authorized Charter School and Local District Percents Proficient in Mathematics; Grades 3-8; 2012 89 percent of SUNY’s charter schools outperform their district of location in mathematics. 65.6 percent of SUNY’s charter schools outperform their local districts in mathematics by at least 20 percentage points. Effect Size Comparison A school’s Effect Size is an important measure for determining the effectiveness of the school’s instruction and provides a basis for comparison between New York’s charter school authorizers. The effect size is a comparative measure that quantifies a school’s performance on the state’s English language arts or mathematics exam while controlling for the percentage of students in each school who are eligible for the federal Free Lunch Program. Inclusion in the Free Lunch Program serves as a proxy measure for poverty, which correlates strongly with academic performance. Each year, the SUNY Charter Schools Institute calculates Effect Sizes in ELA and math for every school authorized by the SUNY Trustees and for all schools throughout the state. Figure 3 shows the aggregate Effect Sizes for each authorizer in New York looking back to the 2008-09 school year.1 The reference line shown in red indicates the threshold for meaningful effect, which is the standard that the Institute expects for each school that it authorizes. Figure 3: ELA and Mathematics Effect Sizes by Authorizer; 2008-09 through 2011-12 Charter schools authorized by the SUNY Trustees have performed higher than expected during each year looking back to the 2008-09 school year in both ELA and mathematics. After the 2008-09 school year, the SED recalibrated the proficiency cut scores for the New York State Testing Program; thus, every school demonstrated a drop in the percentage of students scoring proficient in both ELA and mathematics. Notwithstanding this drop in proficiency levels, SUNY schools posted meaningful Effect Sizes overall in both ELA and mathematics in 2009-10. In ELA, SUNY charter schools demonstrated improvement throughout the school years 2010-11 and 2011-12. In math, SUNY charter schools demonstrated continued strong performance throughout the school years 2009-10, 2010-11, and 2011-12. 2011-12 Effect Size Scatter Plot of SUNY Authorized Charter Schools The school’s ELA Effect Size (on the horizontal axis) and the school’s mathematics Effect Size (on the vertical axis) determine the position of each school on the scatter plot shown in Figure 4. The chart includes schools authorized by the SUNY Trustees. Schools under consideration for charter renewal during 2012 are highlighted in orange. The size of the point representing each school shows the number of students tested during the 2011-12 school year. Compared to demographically similar schools throughout the state, a school that performs higher than expected on the state’s ELA and math exams appears on the chart in the upper-right quadrant. A school that performs lower than expected appears in the lower left quadrant. Schools with relative weaknesses in ELA or math appear in the upper left quadrant or the lower right quadrant, respectively. Figure 4: ELA and Math Effect Sizes for all SUNY Authorized Charter Schools; Grades 3-8; 2011-12 1 Free Lunch statistics are not yet available to the Institute for the 2011-12 school year. The 2011-12 Effect Size estimates employ Free Lunch data from the 2010-11 school year. The estimates are robust because Free Lunch percentages at the school level are relatively stable year over year. --------------- ------------------------------------------------------------ --------------- ------------------------------------------------------------ 1