SUNY Charter Schools Institute and New York State Education Department April 25, 2012 MEMORANDUM To: Members, SUNY Trustees’ Education, College Readiness and Success Committee Senior Vice Chancellor Johanna Duncan-Poitier From: Susan Miller Barker, Interim Executive Director, SUNY Charter Schools Institute Sally Bachofer, Assistant Commissioner, New York State Education Department Subject: Enrollment and Retention Targets This memo outlines the methodology and the process for review, consideration and final adoption by the Board of Regents and the SUNY Board of Trustees of new enrollment and retention targets for public charter schools per 2010 amendments to the Charter Schools Act. In late May 2010 the Legislature and the Governor enacted legislation that made New York eligible for funding under the federal Race to the Top initiative. Foremost in that legislative package were two items – an increase in the number of public charter schools that could be authorized in New York from 200 to 460 – and an agreement to move forward on a new teacher and leader evaluation and support system. As part of this legislative action, the Governor and the Legislature also amended the Charter Schools Act to require public charter schools to enroll and retain Students with Disabilities, English Language Learners, and students eligible for the Federal Free and Reduced Price Lunch program in proportions that are comparable to their local district public schools. The statute says these enrollment and retention (E&R) targets – which all charters will have to meet or exceed – shall be set by the Board of Regents and the SUNY Board of Trustees. For the past 18 months, staff from the New York State Education Department’s Charter School Office and the SUNY Charter Schools Institute have been collaborating to implement this statutory requirement. Staff’s goal is that the underpinnings of targets set by the Board of Regents and the SUNY Board of Trustees – the data considered, data analytics performed, and methodology proposed are all the same – regardless of whether a public charter school has been authorized by the Board of Regents, the SUNY Board of Trustees or either of the two local school districts that have actively chartered schools through 2010. Our goal has been to create a methodology that is clear, understandable and true to the statutory requirement. We also worked to create a methodology that was fair to schools to which the law applies as well as the children and families those schools seek to serve. We believe our recommendations achieve these goals and are well balanced among the many interests and objectives behind the requirement. Specifically, we believe our work has created a proposed methodology that is: a) Statistically valid; b) Accurately portrays the enrollment and retention rates of each district where a charter school is located; c) Recognizes the analytical challenges of comparing individual charter schools to entire school districts; and d) Actionable by New York State’s charter authorizing entities. In creating this methodology, our two agencies have worked to make the best of a difficult matter. Enrollment and retention of students in these classifications has been a controversial issue and the subject of considerable media coverage and discussion both in New York and nationally. In its most simple terms, the issue breaks down as follows: On the one hand, many public charter schools have student bodies that do not reflect their district averages with regard to Students with Disabilities and English Language Learners. When it comes to students eligible for the Free and Reduced Price Lunch program, charters generally match or exceed district averages. On the other hand, public charter schools are required to accept all comers without preference, except for siblings of current students, and must accept students by lottery when applications exceed available seats. Also, charter schools by law may not access at least two strategies they believe would improve their ability to serve Students with Disabilities and English Language Learners – contracting with BOCES and forming cooperatives that allow charters to pool resources in a way that enables students be educated at shared locations. Legislation would be needed to open those two avenues. The Methodology A detailed description of the proposed methodology is attached to this memo. Put in most succinct terms, an individual charter school’s E&R targets will be set against its local school district average of Students with Disabilities, English Language Learners, and students eligible for the Federal Free and Reduced Price Lunch program at the corresponding grade levels. Each charter school will be given an “error band” created to allow for statistical factors related to sample size (much like the “margin of error” you see in statistical sampling). The larger the charter school’s enrollment (i.e., the larger the sample), the smaller the error band, and vice versa. In the case of New York City, charters will be compared to their local Community School District (CSD), except at the high school level (due to citywide high school choice) and for Students with Disabilities (since placement into special education programs is done citywide), which will be compared to the district as a whole. While the Trustees and the Regents would establish targets for charter schools based on a common methodology, each authorizer would separately build in accountability decision-making points, including but not limited to assessing school’s “good faith effort” to making progress towards and meeting their targets. This is proposed because of the practical realities of charter school operation. For example, many schools do not have many vacancies and may reach or get near their targets for incoming students but still not meet their overall school target. We don’t believe it would be fair, or legal, for charters to expel already enrolled students to make room for other students to meet this new E&R target. To help charters understand their specific targets, we are creating a spreadsheet calculator – similar to an on-line “mortgage calculator” – that will allow any school to input its overall enrollment number and its local school district or CSD and see its overall targets in each category. Along with district averages, we have compiled building-specific numbers in each category for every district and charter school in the state, using Basic Educational Data System (BEDS) data. When fully available, this information will allow for comparisons not only between charter schools and their districts of location, but also between charter schools and individual district schools in their neighborhoods. Staff is studying two potential ways that schools could meet or exceed the Enrollment and Retention targets: 1. By meeting their effective targets (i.e., at least the lower end of statistical error band) in each of the three categories for both Enrollment and Retention, or 2. By meeting a “blended target” that equals the sum of the three targets, without any statistical error bands. It will be the ultimate decision of each authorizing entity to decide whether a school under its purview has met or exceeded its targets and any action to be taken regarding a charter school that fails to meet its targets. We have reviewed our proposed methodology against the five other states that currently require similar E&R targets (MA, LA, NC, SC and RI). Our analysis is that our methodology is more nuanced, more complete and more specific to the spirit of New York State’s law. In other words, we believe our proposed methodology is more fair to all involved – i.e., charter schools, districts, students and families – and provides a stronger lever to monitor whether charter schools, and for that matter all public schools, are serving populations representative of their districts. School Oversight As staff, we would review the data at multiple stages in the life of a charter: 1. At application to see if a proposed school’s enrollment accounts for the statutory requirement; and whether its outreach and student marketing program is reasonable in that context. 2. During annual reviews and annual reports, to see if schools have made efforts to meet the enrollment and retention targets. 3. At renewal, analyzing a school’s performance against its targets and its actions to attempt to meet those targets. Timetable The timetable going forward is as follows: • May 7 – Information on E&R methodology posted to SED/SUNY websites. • May 9-10 – Staff presentation to SUNY Board of Trustees or ECRS Committee. • May 10 – Public comment period opens. • May 11 – Informational webinar for interested parties. • May 21-22 – Staff presentation to Regents P-12 Committee. • May 29 – Public comment period closed. • June 11-12 – SUNY ECRS Committee meeting for discussion and vote on proposed methodology. • June 18-19 – Board of Regents P-12 Committee meeting for discussion and vote on proposed methodology. Thank you for your efforts and interest in this matter. As always, our staffs are available to you for questions, comments, concerns, etc. Attachment: Charter School Enrollment and Retention Targets Technical Report - April 25, 2012 SUNY Charter Schools Institute and New York State Education Department Charter School Office This report describes the current proposed methodology behind the calculation of charter school enrollment and retention targets for free- and reduced-price lunch eligible students, limited English proficient students, and students with disabilities as required by New York Education Law Section 2851(4)(e). This report also discusses a proposed way to apply those targets to charter schools. Calculating Enrollment Targets Enrollment targets are based on student demographic and school enrollment information from New York State Education Department student-level data files. Students who enrolled in a traditional district or charter school in the 2010-11 school year were included in the target setting sample. Students explicitly omitted from the target sample include those enrolled in: a) Boards of Cooperative Educational Services (BOCES) schools, b) private schools that send information to NYSED, and c) District 75 in New York City. Because the timing of student classification differs across schools and districts, and to ensure that schools are not “penalized” for declassifying students over time, students were identified as free- or reduced-price lunch students, limited English proficient students, or students with disabilities if they were ever classified as such in the NYSED student-level data files in school years 2009-10, 2010-11, or 2011-12. This approach, by definition, gives credit to schools and districts that enrolled classified students at any point during the most recent three school years. After restricting the target sample to the population of interest, students were assigned to their school and district of attendance on October 1, 2010. Sample targets were also tested using observed enrollments on April 1, 2011 and June 30, 2011, and on these dates in other school years. These analyses validated the use of October 1st enrollments for setting targets since alternative enrollment snapshots yielded comparable targets. Enrollment targets were created through the following two-step process: The number of students from each target group in each potential grade span was summed at the district level, and then divided by the total number of students enrolled in the district and grade span. This yielded a unique proportion of classified students for every district and grade span configuration. In most instances, targets were set as the unique proportion of classified students calculated in the previous step. Targets in New York City Geographic Districts were established differently for students with disabilities, and for all targets when grade configurations spanned high school grades. In such instances, targets were set as the proportion of classified students enrolled citywide to reflect New York City’s student assignment mechanism, through which high school students have citywide school choice and students with disabilities can be assigned to schools that meet their needs through a central assignment process. Additionally, statistical requirements for calculating confidence intervals of large sample proportions required that targets only be calculated when at least 30 total students were enrolled in the district/grade span combination, of which at least 10 were classified into the target group and 10 were not classified. Calculating Retention Targets The process for establishing retention targets involved starting with the same set of student-level data files as for the enrollment targets, and also employing the same initial sample restrictions, i.e. restricting the target sample to include only students who enrolled in traditional district or charter schools. Students were also identified as free- or reduced-price lunch students, limited English proficient students, or students with disabilities if they were ever classified as such in the NYSED student-level data files in school years 2009-10, 2010-11, or 2011-12. After restricting the target sample to the population of interest, retention targets were created through the following three-step process: The number of students from each target group in each potential grade span who experienced a Type 2 discharge between October 1st of a given school year and September 30th of the subsequent school year was summed at the district level, and then divided by the total number of classified students enrolled in the district and grade span between October 1st and June 30th of the given school year. This yielded a unique withdrawal rate for classified students for every district and grade span combination. The withdrawal rate was then subtracted from 100%, corresponding to the total proportion of students initially enrolled, to obtain a unique retention rate for classified students for every district and grade span combination. Retention rates were separately calculated for each district from 2009-10 to 2010- 11, and from 2010-11 to 2011-12. A 2-year average retention rate for each district was calculated by averaging the rates from 2009-10 to 2010-11, and from 2010-11 to 2011-12. With the exception of Albany City School District, Hempstead Union Free School District, and Middletown City School District, the 2-year average was used in place of a single year retention rate because enough variation was observed in school-level rates across years to justify averaging to achieve more precise retention targets. Retention targets for Albany City School District and Hempstead Union Free School District were based solely on retention from 2009-10 to 2010-11 because of data quality concerns in the most recent school year that resulted in systematically lower retention rates in the district from 2010-11 to 2011-12. Conversely, retention targets for Middletown City School District were based exclusively on retention from 2010-11 to 2011-12 because of data quality concerns in the earlier period that resulted in systematically lower retention rates in the district from 2009-10 to 2010-11. In most instances, targets were set as the retention rates calculated in step 2. Targets in New York City Geographic Districts were established consistent with the enrollment methodology described above. Additionally, statistical requirements for calculating confidence intervals required that targets only be calculated when at least 30 classified students enrolled in the district, of which at least 10 experienced Type 2 discharges and 10 did not experience Type 2 discharges. Calculating Standard Errors and Lower Limits (“Effective Targets”) Fluctuations in student populations and sampling frames are likely to yield natural variation in school-level enrollment and retention rates in any given year. A lower limit, or “effective target,” calculated for each target, accounts for this expected variation and thus reflects the lowest possible enrollment or retention rate a school must report to meet its target. The effective target is obtained in a two-step process. The first step requires calculating the standard error (SE) of the target, a metric which captures expected variation as a function of the size of the student body of interest at each school. Smaller schools will have larger standard errors than bigger schools, and therefore be allowed more flexibility in meeting the target, since fewer students contribute to school-specific rate calculations. Because enrollment and retention targets are proportions (i.e. targets assume values between 0 and 1), the standard errors are calculated using the formula for binomial proportion distributions: In the second step, the effective target is obtained by calculating a one-sided 95% confidence interval around the target. A one-sided interval is used instead of a more traditional two-sided interval because the question of interest is whether schools meet or exceed the target, not whether schools fall within the upper or lower bound intervals of the target. The formula for calculating the effective target is: Effective Target=Target-(1.645*SE) where the standard error is calculated as described above, and the multiplier on the standard error (1.645) is the z-score for a standard normal probability distribution corresponding to the target having no more than 5% likelihood of falling below the effective target.