The Massachusetts Board of Elementary and Secondary Education
Charter Schools - Discussion and Vote on Recommendation to Revoke Charter of Dorchester Collegiate Academy Charter School
In December 2015, the Board discussed my recommendation to revoke the charter of Dorchester Collegiate Academy Charter School (DCACS) based on the school's lackluster academic success and its failure to meet the condition-to demonstrate significant and sustained academic improvement-that the Board imposed in 2014 when it placed the school on probation. After extensive additional deliberation, fact gathering, and discussions with DCACS stakeholders-including parents, teachers, administrators, and board members-I continue to believe that the facts support revocation of the school's charter.
My recommendation is based on the fact that DCACS has been a low performing school with high rates of student attrition since its inception. While the school claims to serve a population of students with substantially greater social and emotional needs than is typical, there is no data to support this assertion. Although serving students with special needs is central to the school's mission, the school has failed to maintain a consistent staff with expertise in serving special needs students. Further, while initial program improvement elements were implemented following the probationary designation, the school has been unable to sustain some of these (e.g., instructional coaches) due to faculty instability. In sum, these observations leave me with little confidence that the school has the leadership capability or capacity to place DCACS on a path of substantial and continuous improvement.
My original recommendation as outlined in my memorandum dated December 9, 2015 (Attachment G) was based on my assessment of the totality of the evidence about the school's historical track record of performance, with a particular focus on whether or not the school was able to meet the key probationary condition of significant and sustained academic improvement, and great concern about the high levels of student attrition which began prior to the imposition of probation. As a reminder, the Department of Elementary and Secondary Education (Department) has always placed paramount importance on improvement of student achievement for all student groups as outlined in the Charter School Performance Criteria.1
In addition, charter school regulations outline eight grounds for revocation, including, but not limited to, two of which apply in this case:
The Board may suspend or revoke a charter for cause including, but not limited to: (a) lack of evidence of academic success; . . . or (h) failure to fulfill any conditions imposed by the Board in connection with the grant or renewal of a charter.
603 CMR 1.12(3).
Since our December meeting, my staff and I have devoted a considerable amount of time and thought to reconsidering the data and my recommendation, in order to determine the best way forward. Our reconsideration included the actions that follow. These are addressed in more detail below.
- I carefully reconsidered the evidence of academic success and attrition which formed the basis of my original recommendation.
- I considered additional analysis of evidence requested by the Board and presented by the school about the student population served by the school.
- I reflected upon the Department's accountability processes with respect to qualitative versus quantitative evidence.
- I conducted an unannounced school visit with Associate Commissioner Cliff Chuang on the afternoon of Monday, January 11, 2016, to observe instruction.
- I attended a meeting with approximately 50 parents and teachers at DCACS on the evening of Monday, January 11, 2016, to hear directly about their experiences at the school.
- I have held frequent and ongoing conversations with school leaders and board members to clarify data, discuss how the school is facing its current challenges, and to consider possibilities that would yield a strong likelihood of producing much stronger performance should the school continue to operate beyond June 30, 2016.
1. Reconsideration of Evidence of Academic Success and Attrition
Condition 8
Department staff carefully reconsidered whether there is evidence that DCACS has demonstrated academic success as required by probationary Condition 8:
By September 30, 2015, DCACS must demonstrate that it is an academic success by providing evidence that the school has met or is making substantial progress toward meeting benchmarks in its approved Accountability Plan and, in particular, has demonstrated significant and sustained academic improvement in mathematics and English language arts.
School Percentile
At a high level, the state accountability system quantifies overall school performance through the assignment of a school percentile, comparing performance of similar school types by aggregating measures of student proficiency and growth as measured by performance on mathematics, English language arts (ELA), and Science MCAS exams over four years, with the most weight given to the most recent year.2 As noted in my memorandum of December 9, by this overarching measure, DCACS has not demonstrated improvement compared to schools with similar grade configurations since it was placed on probation. DCACS was at the 23rd percentile when it was placed on probation during School Year (SY) 2013-2014, declined to the 20th percentile in fall 2014, and returned to the 23rd percentile in fall 2015.
Progress and Performance Index
The percentile is a normative metric, so the Department also uses the cumulative Progress and Performance Index (PPI), which encompasses the same indicators as the school percentile but measures a school's progress in narrowing its proficiency gaps against its own targets. For a school to be "on track" toward narrowing proficiency gaps, the cumulative PPI for both all students and the high needs subgroup must be 75 or higher. Reviewing DCACS's overall performance by this measure, the school's cumulative PPI was 73 for all students3 when it was placed on probation and it has continued to fall short of the cumulative PPI target in the most recent 2 years.
| Cumulative PPI | 2013 | 2014 | 2015 |
|---|---|---|---|
| All students | 73 | 63 | 63 |
| High Needs | NA | 51 | 58 |
Mathematics and English Language Arts
The Department also re-examined whether there was evidence of "significant and sustained academic improvement in mathematics and English language arts." Setting aside the school's performance in Science, which is very low, the school does demonstrate some improvement between SY134 and SY15 on MCAS mathematics and ELA exams as measured by Composite Performance Index (CPI) and Student Growth Percentiles (SGP) in both subjects, though no clear upward trend was established over the two years. As outlined by the school in its December 14 memorandum to you, I also acknowledge the relatively stronger results achieved by the second 8th grade cohort at the school in SY15 compared to its inaugural 8th grade cohort in SY14. Even when restricting analysis of performance to mathematics and ELA, however, I do not believe the evidence can be characterized as significant or sustained with respect to either CPI or SGP measures. This is especially true because the school's improved performance in SY15 is essentially the same as its performance in its second year of operation in SY11 on both sets of measures in both subjects.
Cohort Performance
In order to assess the school's assertion that students make more progress after becoming acclimated to the academic and behavioral expectations at DCACS, the Department examined the performance of the first two 8th grade cohorts over time-the SY10 inaugural cohort that took 8th grade MCAS exams in SY14 and the SY11 cohort that took 8th grade MCAS exams in SY15. See Attachment I: Cohort Performance Over Time. The results presented in Attachment I are inconclusive regarding the school's claim of improved performance over time, both due to the small number of students5 and the variability in results. Out of the SY10 cohort of 40 students, 14 took the 8th grade MCAS in Spring 2014; for the SY11 cohort of 30, 7 took the 8th grade MCAS in Spring 2015. Comparison information for other Boston charter schools and BPS is also presented, though this should also be viewed with caution due to varying grade span structures (in some cases students attend the same school in grades K through 8), and the fact that many schools took PARCC in 2015.
Comparative student performance
In my memorandum of December 9, I summarized DCACS's overall comparative performance as follows:
Over the seven-year duration of the school's history, DCACS's combined CPIs for all grades, in all three subject areas, have been the lowest, or among the lowest, of all the charter schools in Boston, and are well below state averages. Over this same time period, when compared to the CPI of the Boston Public Schools at the same grade levels, DCACS falls somewhere in the middle of the range of scores in ELA and science, with generally lower performance in mathematics.
To illustrate this comparison and in response to Board member inquiries, a graphic representation of this information is provided in Attachment F: Academic Performance Comparison to other Boston Schools. In addition, Attachment F provides a comparison of the relative performance of DCACS in relation to other BPS and Boston charter schools, restricted to the same school-type, based on school-percentile and cumulative PPI. The Department's assessment of the school's comparative performance remains generally the same, even when viewed using these alternate measures.
Accountability Plan
The Department re-examined the school's performance as articulated in its Accountability Plan, through which the school is able to articulate unique measures that demonstrate success as aligned to the school's mission and charter. In its guidance documents, the Department strongly encourages schools to develop and report alternate measures of student success in their accountability plans, particularly if they believe that their student population is unique or that the standard metrics used by the Department (e.g., MCAS) may not accurately reflect the school's performance. DCACS's Accountability Plan for its second charter term was approved in April 2015 and the school first reported on its performance on those measures in its 2014-15 Annual Report. DCACS did not articulate any measures meant to demonstrate student academic success or to demonstrate the unique composition of its student population in its Accountability Plan. There are two measures related to developing a school wide literacy program which track student performance on school-created open response assessments and lexile levels. The school reported in its recent Annual Report that it met both measures.
Attrition Rate - Updates to Data Presented in December
As noted in my memorandum of December 9, attrition rates6 for all students and all subgroups have been high over the past several years. Since December 9, the Department's Educator Data Services unit completed the reconciliation of duplicate October 1, 2015 claims for students in the Student Information Management System (SIMS), and it was determined that the school retained 8 fewer students over the summer of 2015 than originally reported. This means that the aggregate attrition rate for 2015-2016 is actually 18.5 percent rather than the previously-reported 14.4 percent, which, though still improved from the 30.2 percent attrition rate for 2014-2015, remains among the highest of all Boston schools serving similar grades. See Attachment E: Charter Analysis and Review Tool (CHART) Attrition Data for an updated graph presenting this information.
2. Additional Analysis of Information Regarding Student Populations and Performance
During the Board meeting on December 15, 2015, Board members asked for additional comparative data regarding DCACS's student population. Department staff conducted additional analyses of DCACS's enrollment trends, with a particular focus on students with disabilities, from school year SY10 to SY16. Attachment A, Subgroup Enrollment SY10-SY16, shows DCACS's certified enrollment of students with disabilities, low income/economically disadvantaged, and English language learners (ELL) during the three SIMS collection points throughout the year (A (October), C (March), and D (June)). Attachment A also shows the comparative total enrollments of students with disabilities at other Boston charter schools and in Boston Public Schools (BPS).
School leaders explained that staff likely reported and certified erroneously coded data during SY13 and SY14, as indicated by the dramatic shifts in several subgroup populations between SIMS A and C collection points.7 Please see Attachment D: CHART enrollment data to see a graphic representation of comparative enrollment of DCACS's subgroup enrollment compared to other Boston charter and district schools. This data display has been updated since December 9 to include data for SY16, as well as modified to provide the highest percentage of each subgroup reported by DCACS at any SIMS collection within a given school year.
Students with Disabilities
From SY10 through SY12, DCACS enrolled a stable population of students with disabilities throughout the entire academic year at rates lower than both BPS and Boston charter schools. Beginning in SY13, DCACS began enrolling a more comparable percentage of students with disabilities, but the increased numbers are not reflected in the SIMS A collection number in SY13 or SY14 due to data reporting/coding errors as submitted to SIMS and certified by the school leader. The school indicates that they resolved their data reporting during SY14 by increasing capacity in this area, though reporting errors continued until this fall (SY16) with respect to students with disabilities. Even with the data corrections supplied by the school this week, I remain unconvinced that the school's historical student population has been substantively different than that of other Boston district or charter schools. Below is evidence to support my assessment.
Disability Level of Need8
Attachment B: Level of Need SY10-SY16 displays the level of need for students with disabilities as reported through SIMS for students at DCACS, in BPS, and at other Boston charter schools. From the school's first year of operation in SY10 and through SY15, DCACS has only reported enrolling students with disabilities who have low levels of need, with a vast majority of the students requiring fewer than 2 hours of services per week. During the school's first six years of operation, DCACS did not report enrolling any students with moderate or higher levels of need, in stark contrast to other Boston charter schools and BPS schools.
After a number of conversations between Department staff and the school's headmaster and special education director, the school recently reported that DCACS enrolled students with moderate levels of need beginning in SY13 and moderate and high levels of need in SY14. While the new calculations provide a picture of more comparable levels of need, in SY13 through SY15 DCACS still enrolled a lower percentage of moderate level of need when compared to other Boston charter schools and BPS schools. The percentage of students with a high level of need is comparable to Boston charter schools and lower than BPS schools. See Attachment H for the school's own calculations of level of need from SY13-SY16.
Disability Type
Attachment C: Disability Type SY10-SY16 outlines the number/percentage of students with a particular disability at DCACS, in BPS, and at other Boston charter schools. Due to the relatively small numbers of special education students enrolled at DCACS, particular through SY13, this data should be considered with appropriate caution. The majority of DCACS's students with disabilities have specific learning disabilities, which is true for BPS and Boston charter schools. DCACS, however, for the first six years of its operation, had not enrolled a single student with autism, a neurological disability, a physical disability, or who is hard of hearing/deaf or vision impaired/blind. This contrasts to BPS and Boston charter schools. Additionally, DCACS has enrolled relatively few or no students (0-4 per year) who have multiple disabilities, an intellectual disability, or who have a developmental delay. DCACS, BPS, and Boston charter schools have enrolled similar percentages of students with a communication disability or a health disability. In the past three years, DCACS has had similar rates of students with an emotional disability when compared to BPS (approximately 2-3 percent) and at slightly greater rates than Boston charter schools (1.2-2.2 percent). The actual number of DCACS students with an emotional disability has ranged from five to six students over the past three years. Currently, the school enrolls six students with an emotional disability.
Out-of-district placements
I also want to clarify what the law requires with respect to students with disabilities who require out-of-district placements in a day or residential setting. Charter schools are required to comply fully with the state special education law, G.L. c. 71B, including making available a full continuum of in-district special education services. This is true with one exception: the fiscal and programmatic responsibility for a special needs student currently enrolled in or determined to require an out-of-district program remains with the school district where the student resides.9 This applies to DCACS and all other charter schools and provides a partial explanation as to why students with a high level of need are represented at higher rates in BPS than in Boston charter schools, as noted in Attachment B.
Counseling services
In the school's letter to the Board dated December 14, 2015, school representatives stated that over 45 percent of the school's students require counseling services. This is the first time the school has reported such information. The school has not outlined any related measurable goals in its Accountability Plan or provided other data to demonstrate that it serves a specialized student population with respect to social-emotional needs. The Department does not independently collect this type of data, and we have asked the school to provide additional information about how it documents and tracks this information, prior to your meeting on January 26, 2016. At the meeting, school representatives will also have the opportunity to provide more context for this data point and to answer questions from Board members directly.
3. Qualitative versus Quantitative Evidence
I want to comment on the qualitative evaluation performed by Department staff as documented in the May 2015 Year Six Site Visit Report, which indicated that the school met expectations with respect to mission and program delivery. After further discussion with staff, I believe that the report does accurately describe what the site visit team observed on the day of the site visit, recognizing the programmatic improvements made by the school since renewal that appear to have resulted in some improvement in mathematics and ELA as noted above. The site visit team only examined the degree to which the school was meeting its mission in the context of implementing an academic program that aligned with its charter and subsequent amendment, as required by the conditions. The team did not examine or gather evidence pertaining to other mission driven aspects of the school. That said, it is important to underscore that this type of qualitative data is designed to provide additional context about a school's program but cannot replace the more comprehensive measures of student performance provided by the state's assessment system.
4. School Visit on January 11, 2015
Associate Commissioner Chuang joined me on an unannounced visit to DCACS on the afternoon of January 11, 2015, to observe instruction. We observed each of the school's 10 classrooms (2 at each grade level, 4th through 8th), with the exception of a 4th grade physical education class conducted outside. I am fully aware that the short duration and informal nature of our observations cannot replace a formal school visit-like the May 2015 site visit at DCACS-that the Department conducts with full teams using rubrics and protocols. That said, I wanted a personal, unvarnished, and unannounced look at the current state of the academic program.
Based on my observations, in general, the school remains orderly. Instructional quality, however, was inconsistent. I observed several classes with low levels of academic expectations and a class in which a concerning lack of content knowledge was exhibited by the instructor. I did not see any classes that I would characterize as demonstrating strong instructional practice. The curriculum, including texts that students were reading, was typically of relatively low level with students primarily engaged in finding surface level facts; there was little higher order thinking expected of students.
The school explained that it has had to adapt to the loss of 4 teachers (out of approximately 20) within the last month, precipitated by my December recommendation to revoke the school's charter, which has stretched the school's capacity. I fully acknowledge the additional stress and anxiety that my recommendation may have provoked. I am very concerned, however, by how quickly the school appears to have destabilized, and I have little confidence that the school's current academic program will result in significant improvement in student outcomes.
5. Meeting with Parents and Teachers on January 11, 2015
Associate Commissioner Chuang and I also met with approximately 50 parents and teachers on the evening of January 11, 2016, to share the basis for my December recommendation and to hear their thoughts and concerns directly. All parents and teachers who were present advocated for keeping the school open. It is clear that the school has substantial support from an impassioned group of parents who appreciate the individual attention their students receive with respect to both academics and social-emotional health and a safe school culture free from bullying. Parents universally spoke about how much they value their ability to choose a public charter school that best fits their child's needs and asked me to consider the success and progress that their individual children have achieved at DCACS. Teachers spoke about how much work they had put into making the programmatic improvements required by the probationary conditions.
The Department has received correspondence from DCACS stakeholders regarding the revocation recommendation, and it is being provided to Board members under separate cover.
6. Additional Discussions with School Officials about a Path Forward
Throughout the past several weeks, Department staff and I have engaged in frequent and ongoing conversations with DCACS officials about data, and we openly considered the possibilities for a potential path forward to provide a strong program for students after June 30, 2016. Unfortunately, despite good intentions on the part of school officials, these discussions have only heightened my concerns about the school's capacity to provide a strong program. Of particular note are the concerns that follow.
I am not confident that students with disabilities and English language learners are receiving the support they require. In SY16, the current school year, DCACS has reported its highest levels of students with disabilities (27.6 percent) and a high percentage of ELLs (20.1 percent), which, if accurate, are higher than the average rates of other BPS and Boston charter schools. Additionally, the school has reported its highest levels of special education services needed this year, with 16 percent of students requiring low levels of services, 8 percent requiring moderate levels of services, and 2 percent requiring high levels of services. Of particular concern, DCACS planned to employ five special education teachers and a full-time special education coordinator in the current school year. Currently, the school employs three special education teachers and a part-time special education coordinator. The school has not filled two of the vacancies that occurred during the summer of 2015. All but one of the special education staff, a teacher, are new to the school in SY16. The school does not currently employ an ELL coordinator and, additionally, it is unclear if the school's current ELL teacher is certified.
While the school's headmaster has served in that capacity since the school's founding, the school has lacked consistent academic leadership, and current academic leadership is compromised by staffing vacancies. In its first charter term, DCACS did not consistently employ an individual responsible for providing teachers with adequate observation, feedback, or evaluation. In its fifth year of operation, the school employed part-time consultants to provide teachers with guidance and feedback. As documented in the May 2015 Year 6 Site Visit Report, in the school's sixth year (SY15), the school employed two full-time instructional coaches, which appears to have led to some programmatic improvements. Both of these individuals currently remain at the school as subject area deans and provide instructional leadership. Because the school has lost 4 teachers out of approximately 21 over the last month, the ability of these instructional coaches to provide support to teachers is compromised, requiring the headmaster to provide some of that coaching support, further straining administrative capacity. Teacher attrition within this year is in addition to a loss of 5 teachers over the summer and continues a trend of low teacher retention, with 52.6 percent of the prior year's teachers currently employed at DCACS. Additionally, rates of teacher attrition have been very high in previous years; please see the table below.
| DCACS teacher retention rates | A: Total number of teachers end of prior school year | B: Total number of teachers retained (remain employed as of beginning of next school year) | C: Percentage of teachers retained (B/A = C) |
|---|---|---|---|
| 2012-13 | 10 | 4 | 40% |
| 2013-14 | 13 | 7 | 53.8% |
| 2014-15 | 17 | 9 | 52.9% |
Recommendation
In short, I have little confidence that the school is poised to provide a strong program for its students. DCACS has been a low performing school with high rates of student attrition since its inception. While the school claims to serve a population of students with substantially greater social and emotional needs than is typical, there is no data to support this assertion. Although serving students with special needs is central to the school's mission, the school has failed to maintain a consistent staff with expertise in serving special needs students. Further, while initial program improvement elements were implemented following the probationary designation, the school has been unable to sustain some of these (e.g., instructional coaches) due to faculty instability. In sum, these observations leave me with little confidence that the school has the leadership capability or capacity to place DCACS on a path of substantial and continuous improvement. For these reasons, I affirm my original recommendation that the school's charter be revoked as of June 30, 2016.
If the Board votes its intent to revoke the school's charter in accordance with 603 CMR 1.12(4), the school will have 15 days to request an administrative hearing pursuant to M.G.L. c. 30A, § 13, and 801 CMR 1.00: Standard Adjudicatory Rules of Practice and Procedure, if it chooses to do so.10 If the Board does not receive a request for a hearing from the school within the 15-day period, the Board's conditional action on revocation of the charter shall become final at the end of the 15-day period.
Department staff have been working with the school and BPS and Boston charter school officials to schedule an enrollment information session on the evening of January 26 to ensure that the families are aware of all their public school options for the next school year. DCACS representatives have been invited to address the Board at the January 26 meeting and have chosen to do so. Department staff will also be available to present information and answer questions.
If you have any questions regarding this recommendation or require additional information, please contact Cliff Chuang, Associate Commissioner (781-338-3222); Jeff Wulfson, Deputy Commissioner (781-338-6500); or me.
Attachments:
1 Charter School Performance Criteria.
2 For each school with valid data, the Department (a) calculates percentile ranks (1-99) for each achievement, improvement, and high school indicator as compared to other schools of the same school type, (b) calculates a mean (average) rank across each of the achievement and improvement indicators that places progressively more weight on data from more recent years (i.e., 1-2-3-4), (c) standardizes the relative value of the achievement and improvement means within each school type category so that they are comparable, and (d) combines these means, with the achievement mean weighted more heavily than the improvement mean.
3 In 2013, the school did not have sufficient data for high needs students to calculate a cumulative PPI for that subgroup.
4 SY13 scores were the last scores available prior to probation.
5 The small number of students still enrolled in SY2014 from the original SY2010 cohort may be due to both attrition and grade-level retentions.
6 The attrition rate represents the percentage of students who were enrolled at the end of one school year and did not remain in the same school in the following fall. Students in the school's highest grade are not included in the calculation.
7 The Department worked with the school to make a special correction to the low income data submitted during SY14 to ensure that the school was not adversely affected for tuition rates in the following year. The ELL data error, however, did not come to light until this past month.
8 The Student Information Management System (SIMS) Data Handbook defines Special Education - Level of Need ) as "the amount and type of service that a student receives inside and outside of the general education environment as determined by the school district upon review of the student's IEP."
9 The Department has developed extensive guidance on this issue-see Technical Assistance Advisory SPED 2014-5: Charter School Responsibilities for Students with Disabilities Who May Need an Out-of-District Program - 603 CMR 28.10(6) -to assist both charter schools and districts with carefully making such determinations.
10 Upon receiving a notice of intent to revoke a charter, notice of non-renewal, or notice of suspension where the health, safety, or education of the school's students is at immediate risk, the school shall have the rights provided in M.G.L. c. 30A, § 13, and 801 CMR 1.00: Standard Adjudicatory Rules of Practice and Procedure. All requests for hearings shall be in writing, addressed to the Board, and must be received within 15 days of receipt by the charter school of notice. At such hearing, the school shall bear the burden of proof and present its case first. 603 CMR 1.12(5)