SUNY CHARTER SCHOOLS INSTITUTE March 12, 2012 Memorandum To: Members of the Education, College Readiness and Success Committee From: Susan Miller Barker, Interim Executive Director, SUNY Charter Schools Institute Subject: Establishing Enrollment and Retention Targets for Charter Schools Since our last report on this topic, the Charter Schools Institute (Institute) has continued to work with the New York State Education Department (NYSED) to develop enrollment and retention targets as required by the May 2010 amendments to the Charter Schools Act of 1998 (Act). Our work has steadily progressed in recent months, and we anticipate a joint roll-out of the proposed framework to seek comments from the public prior to presenting the targets to the Education, College Readiness and Success Committee (Committee) for adoption. Statutory Framework As discussed more fully in the September 9, 2011, memorandum to the Committee on this topic, the Act requires the SUNY Trustees to establish enrollment and retention targets for students with disabilities (SWDs), English language learners (ELLs), and students who are eligible applicants for the federal Free and Reduced Price Lunch program (FRPL). The Institute, and ultimately the Committee, will evaluate a school against the enrolment and retention targets at three distinct points: - Evaluating proposals to establish a new charter school – the Act states that the SUNY Trustees “shall not consider any [RFP] applications which do not rigorously demonstrate that they . . . would meet or exceed enrollment and retention targets.” - Annual reports – schools must report “efforts taken . . . in the existing school year, and a plan for efforts to be taken in the succeeding school year, to meet or exceed enrollment and retention targets.” - At renewal – schools must report on “the means by which [it] will meet or exceed enrollment and retention targets.” The Institute will also analyze whether the school has met the targets for the preceding charter term, and report the results to the Committee. In their annual reports, schools will need to describe how they have progressed towards meeting their enrollment and retention targets. As schools come to renewal the Institute will provide an assessment of progress toward and accomplishment of the targets. If a school fails to meet its enrollment or retention targets at two or more consecutive renewals, the SUNY Trustees or the Board of Regents may terminate the school’s charter for “repeated failure to comply with the requirement to meet or exceed the enrollment and retention targets, provided that [the school] has not demonstrated that it has made extensive efforts to recruit and retain such students, including outreach to parents and families in the surrounding communities, widely publicizing the lottery for such school, and efforts to academically support such students in such charter school.” Limitations When developing the targets, SUNY must ensure the enrollment targets are “comparable” to the enrollment figures and rates of retention of the selected populations attending public schools within the school district of the charter school’s location. As enrollment of these student populations vary across districts, targets will vary from charter school to charter school based on its district of location, or, in the case of New York City, the community school district (CSD) in which it is located. In trying to develop valid and equitable enrollment and retention targets, the Institute endeavored to adhere to the Act’s requirements, accurately portray the enrollment and retention rates of each district, rely on valid and accepted statistical practices, and recognize the challenges of comparing districts to individual charter schools. Throughout the process of analyzing available data and developing enrollment and retention targets, the Institute and NYSED encountered and attempted to adjust for various systemic limitations. For example, comparing a single school with proportionally low enrollment to an entire school district presents significant challenges, exacerbated in some areas by the lack of available reported data. Even where sufficient reliable data exists, significant limitations still exist, including: - Non-English speakers do not locate evenly across geographic locations. This creates a wide variation of ELL enrollment and retention rates across district schools driven primarily by geography. - Similarly, many districts consolidate the provision of special education services, based on programmatic needs, within specific schools or may contract with an external agency for provision of those services. New York City has created a specific community school district to serve students with special needs, District 75. Many upstate school districts contract with BOCES for the provision of special education services (notably, charter schools are statutorily prohibited from doing so). - For purpose of special education, charter schools are considered schools within the Local Education Agency (LEA) of each student’s district of residence. Thus, districts control whether SWDs are placed in charter schools or a district setting. Further, many charter schools have designed and implemented robust Response to Intervention (RTI) procedures or have enhanced general education settings, which limit the number of students that are initially referred to the LEA for classification as a SWD. - Open and operating charter schools cannot easily influence their student population, as they typically have a small number of openings each year, limited to a particular grade range, and movement into charter schools tends to be restricted during the school year while, at the same time, movement out of district schools tends to be restricted. Developing an accurate aggregate statistic about the district and a school-level metric for comparison has proven extremely difficult given these and other limitations. Overview of the Institute’s and NYSED’s Work Working collaboratively with NYSED and various interdisciplinary consultants, the Institute developed more than a dozen potential iterations of the enrollment and retention targets, resulting in three distinct “models.” With the development of each model, the team scrutinized the model, pressure test the model using actual data, identify weaknesses with the proposed mode, and ultimately use the results to adjust the model. The first model involved calculating the means of each enrollment and retention category by district, using the student as the unit of analysis for enrollment and a school as the unit of analysis for retention. This model was ultimately abandoned because it was heavily subject to influence of “outliers” (e.g., non-representative schools with proportionally large enrollment), and it was not equitable because it did not fully recognize the unique characteristics of schools that comprise a district. The second model used weighting to calculate the proportion of identified students at every possible grade band. While this model accurately portrayed enrollment and retention rates at district level by grade span, it nevertheless failed to account for variability in enrollment related to school size, remained subject to the influence of outliers, and still did not fully recognize the unique characteristics of schools. The current model involves first calculating the proportion of SWDs, ELLs and FRPLs at district level by grade band, similar to the weighted model, then calculating the 95% confidence interval based on the proportion, and setting the target as the lower bound of the 95% confidence interval. While several methodologies were employed to generate confidence intervals, the goal was to attempt to account for variation in subgroup populations between schools as a function of the school’s total enrollment. Under the proposed confidence interval methodology, smaller schools will have a larger “margin of error” than larger schools, due to the higher likelihood of random variation with a smaller sample of students. This model establishes targets at the district level, accounts for the effects of random selection of students into the charter school, and accounts for variability in enrollment rates that is related to overall school size. Next Steps The Institute and NYSED continue to work toward presenting the identified target setting model to the public throughout the state via a webinar. Following the webinar, the Institute will open a comment period to receive feedback. The Institute and NYSED will utilize the feedback to further refine the target setting methodology. Thereafter the draft methodology for setting targets would be brought to the Committee for consideration and potential adoption.